-- Logs begin at Wed 2022-11-02 06:46:07 UTC, end at Wed 2022-11-02 07:47:25 UTC. -- Nov 02 07:15:45.752338 dv-cinder-804735-33-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Nov 02 07:15:45.759562 dv-cinder-804735-33-iscsi devstack@keystone.service[81488]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Nov 02 07:15:45.762166 dv-cinder-804735-33-iscsi devstack@keystone.service[81488]: *** Starting uWSGI 2.0.18-debian (64bit) on [Wed Nov 2 07:15:45 2022] *** Nov 02 07:15:45.762166 dv-cinder-804735-33-iscsi devstack@keystone.service[81488]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Nov 02 07:15:45.762166 dv-cinder-804735-33-iscsi devstack@keystone.service[81488]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Nov 02 07:15:45.762166 dv-cinder-804735-33-iscsi devstack@keystone.service[81488]: nodename: dv-cinder-804735-33-iscsi Nov 02 07:15:45.762166 dv-cinder-804735-33-iscsi devstack@keystone.service[81488]: machine: x86_64 Nov 02 07:15:45.762166 dv-cinder-804735-33-iscsi devstack@keystone.service[81488]: clock source: unix Nov 02 07:15:45.762166 dv-cinder-804735-33-iscsi devstack@keystone.service[81488]: pcre jit disabled Nov 02 07:15:45.762166 dv-cinder-804735-33-iscsi devstack@keystone.service[81488]: detected number of CPU cores: 4 Nov 02 07:15:45.762166 dv-cinder-804735-33-iscsi devstack@keystone.service[81488]: current working directory: / Nov 02 07:15:45.762166 dv-cinder-804735-33-iscsi devstack@keystone.service[81488]: detected binary path: /usr/bin/uwsgi-core Nov 02 07:15:45.762793 dv-cinder-804735-33-iscsi devstack@keystone.service[81488]: your processes number limit is 47680 Nov 02 07:15:45.762793 dv-cinder-804735-33-iscsi devstack@keystone.service[81488]: your memory page size is 4096 bytes Nov 02 07:15:45.762793 dv-cinder-804735-33-iscsi devstack@keystone.service[81488]: detected max file descriptor number: 2048 Nov 02 07:15:45.762793 dv-cinder-804735-33-iscsi devstack@keystone.service[81488]: lock engine: pthread robust mutexes Nov 02 07:15:45.764983 dv-cinder-804735-33-iscsi devstack@keystone.service[81488]: thunder lock: enabled Nov 02 07:15:45.764983 dv-cinder-804735-33-iscsi devstack@keystone.service[81488]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Nov 02 07:15:45.764983 dv-cinder-804735-33-iscsi devstack@keystone.service[81488]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Nov 02 07:15:45.802127 dv-cinder-804735-33-iscsi devstack@keystone.service[81488]: Python main interpreter initialized at 0x55f307e0d9c0 Nov 02 07:15:45.802127 dv-cinder-804735-33-iscsi devstack@keystone.service[81488]: python threads support enabled Nov 02 07:15:45.802127 dv-cinder-804735-33-iscsi devstack@keystone.service[81488]: your server socket listen backlog is limited to 100 connections Nov 02 07:15:45.802127 dv-cinder-804735-33-iscsi devstack@keystone.service[81488]: your mercy for graceful operations on workers is 90 seconds Nov 02 07:15:45.802534 dv-cinder-804735-33-iscsi devstack@keystone.service[81488]: mapped 403077 bytes (393 KB) for 2 cores Nov 02 07:15:45.802614 dv-cinder-804735-33-iscsi devstack@keystone.service[81488]: *** Operational MODE: preforking *** Nov 02 07:15:45.802826 dv-cinder-804735-33-iscsi devstack@keystone.service[81488]: *** uWSGI is running in multiple interpreter mode *** Nov 02 07:15:45.802826 dv-cinder-804735-33-iscsi devstack@keystone.service[81488]: spawned uWSGI master process (pid: 81488) Nov 02 07:15:45.803029 dv-cinder-804735-33-iscsi systemd[1]: Started Devstack devstack@keystone.service. Nov 02 07:15:45.803622 dv-cinder-804735-33-iscsi devstack@keystone.service[81488]: spawned uWSGI worker 1 (pid: 81501, cores: 1) Nov 02 07:15:45.803953 dv-cinder-804735-33-iscsi devstack@keystone.service[81488]: spawned uWSGI worker 2 (pid: 81502, cores: 1) Nov 02 07:15:45.805016 dv-cinder-804735-33-iscsi devstack@keystone.service[81488]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Nov 02 07:15:50.043767 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Nov 02 07:15:50.043767 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: from cryptography.utils import int_from_bytes Nov 02 07:15:50.044563 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Nov 02 07:15:50.044563 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: from cryptography.utils import int_from_bytes Nov 02 07:15:50.082159 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Nov 02 07:15:50.082159 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: from cryptography.utils import int_from_bytes Nov 02 07:15:50.082706 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Nov 02 07:15:50.082706 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: from cryptography.utils import int_from_bytes Nov 02 07:15:50.093464 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Nov 02 07:15:50.093655 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Nov 02 07:15:50.093844 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] command line args: [] {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Nov 02 07:15:50.094108 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Nov 02 07:15:50.094283 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ================================================================================ {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Nov 02 07:15:50.094563 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] admin_token = **** {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:15:50.094803 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] config_dir = [] {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:15:50.094987 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:15:50.095165 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] config_source = [] {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:15:50.095342 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] debug = True {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:15:50.095537 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:15:50.096710 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:15:50.096710 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:15:50.096710 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:15:50.096710 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:15:50.097003 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] list_limit = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:15:50.097003 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] log_config_append = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:15:50.097132 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:15:50.097234 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] log_dir = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:15:50.097403 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] log_file = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:15:50.097637 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:15:50.097883 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:15:50.098123 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:15:50.098389 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:15:50.098561 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:15:50.098735 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:15:50.098974 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:15:50.099214 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:15:50.099444 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:15:50.099674 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:15:50.099902 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:15:50.100129 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:15:50.100371 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:15:50.100608 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:15:50.100877 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:15:50.101353 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.18/identity {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:15:50.101534 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] publish_errors = False {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:15:50.101759 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:15:50.101983 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:15:50.102157 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:15:50.102421 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:15:50.102655 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:15:50.102885 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] standard_threads = False {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:15:50.103110 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:15:50.103345 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:15:50.103517 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:15:50.103689 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] use_journal = False {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:15:50.103861 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] use_json = False {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:15:50.104031 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] use_stderr = False {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:15:50.104201 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] use_syslog = False {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:15:50.104425 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:15:50.104672 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.105035 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.105293 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.105531 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.105777 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.106265 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.106535 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.106769 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.107013 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.107257 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.107506 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.107740 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.107984 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.108219 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.108461 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.108705 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.108939 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.109190 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.109442 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.109684 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.109916 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.110167 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.110446 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.110697 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.110942 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.111200 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.111399 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.111625 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.111856 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.112099 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.112342 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.112569 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.112795 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.113037 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.113277 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.113513 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.113757 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.113986 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.114217 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.114473 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.114701 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.114927 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.115153 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.115379 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.115608 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.115853 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.116082 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.116313 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.116539 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.116768 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.116999 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.117242 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.117469 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.117702 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.117940 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.118173 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.118438 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.118677 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.118903 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.119146 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.119376 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.119607 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.119833 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.120087 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.120341 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.120666 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.120904 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.121236 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.121504 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.121740 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.121998 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.122225 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.122483 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.122719 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.122952 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.123196 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.123431 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.123665 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.123906 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.124138 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.124380 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.124614 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.124845 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.125075 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.125306 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.125537 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.125791 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.126029 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.126272 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.126534 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.126769 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.127008 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.127241 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.127473 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.127707 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.127937 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.128163 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.128388 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.128619 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.128843 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.129078 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.129309 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.129552 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.129787 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.130036 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.130291 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.130554 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.130782 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] auth.external = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.131008 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.131142 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Nov 02 07:15:50.131298 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.131472 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Nov 02 07:15:50.131472 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] command line args: [] {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Nov 02 07:15:50.131735 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.131735 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] auth.password = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.131980 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Nov 02 07:15:50.131980 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] ================================================================================ {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Nov 02 07:15:50.132235 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] auth.token = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.132363 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] admin_token = **** {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:15:50.132495 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.132495 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.132733 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] config_dir = [] {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:15:50.132733 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:15:50.132976 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.133109 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] config_source = [] {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:15:50.133241 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.133369 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] debug = True {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:15:50.133505 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.133639 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:15:50.133799 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.133928 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:15:50.133928 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:15:50.134171 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.134171 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] credential.caching = True {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.134518 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:15:50.134653 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.134779 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:15:50.134905 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.135042 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] list_limit = None {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:15:50.135042 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] log_config_append = None {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:15:50.135294 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.135421 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:15:50.135547 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.135679 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] log_dir = None {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:15:50.135808 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.136439 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] log_file = None {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:15:50.136661 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.136799 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:15:50.136933 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.137060 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:15:50.137328 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.137580 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:15:50.138126 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.138263 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:15:50.138436 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.138573 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:15:50.138703 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.138703 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.138703 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.138703 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.139174 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:15:50.139302 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.139302 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] federation.caching = True {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.139302 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.139700 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:15:50.139850 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.139979 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:15:50.140107 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.140107 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.140355 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:15:50.140483 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.140611 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:15:50.140746 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.140880 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:15:50.141015 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.141154 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:15:50.141282 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.141457 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:15:50.141609 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.141748 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:15:50.141885 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.142014 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:15:50.142151 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.142151 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] identity.caching = True {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.142409 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.18/identity {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:15:50.142544 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.142681 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] publish_errors = False {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:15:50.142812 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.142939 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:15:50.143066 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.143192 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:15:50.143330 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.143464 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:15:50.143593 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.143720 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:15:50.143848 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.143975 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:15:50.144101 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.144235 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] standard_threads = False {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:15:50.144363 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.144490 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:15:50.144617 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.144744 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:15:50.144744 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:15:50.145010 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.145149 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] use_journal = False {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:15:50.145277 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.145403 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] use_json = False {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:15:50.145531 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.145657 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] use_stderr = False {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:15:50.145657 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] use_syslog = False {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:15:50.145911 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.146056 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:15:50.146193 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.146320 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.146485 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.146627 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.146762 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.146900 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.147035 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.147170 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.147298 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.147431 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.147560 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.147687 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.147814 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.147814 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.148059 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.148195 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.148324 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.148452 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.148586 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.148725 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.148853 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.148989 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.149123 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.149251 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.149378 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.149505 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.149710 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.149848 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.149986 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.150116 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.150243 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.150389 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.150538 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.150677 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.150813 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.150942 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.151070 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.151198 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.151331 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.151459 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.152440 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.152585 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.152715 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.152843 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.152970 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.153098 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.153098 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.153363 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.153363 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.153609 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.153743 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.153899 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.154040 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.154170 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.154310 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.154490 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.154620 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.154749 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.154886 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.155032 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.155170 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.155307 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.155437 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.155564 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.155692 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.155819 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.155819 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.156074 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.156202 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.156202 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.156202 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.user = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.156202 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.156685 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.156836 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.156967 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.157095 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.157223 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.157350 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.157486 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.157616 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.157744 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.157872 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.158000 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.158139 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.158139 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.158410 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.158557 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.158687 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.158820 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.158956 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.159085 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.159213 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.159340 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.159468 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.159597 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.159597 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.159848 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.159976 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.160103 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.160231 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.160369 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.160505 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.160635 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.160763 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.160891 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.161019 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.161146 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.161282 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.161411 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.161538 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.161711 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.161841 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.161968 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.162104 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.162243 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.162430 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.162567 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.162697 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.162826 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.162965 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.163103 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.163232 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.163360 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.163488 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.163616 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.163752 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.163881 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.164020 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.164148 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.164276 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.164403 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.164539 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.164673 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.164803 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.164803 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.165063 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.165192 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.165397 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.165530 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.165658 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.165786 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] resource.caching = True {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.165925 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.166054 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.166190 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.166320 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.166476 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.166606 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.166743 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.166881 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.167020 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.167150 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.167307 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.167465 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.167594 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.167779 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.167940 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.168074 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.168203 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.168331 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.168459 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.168587 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] role.caching = True {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.168726 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.168856 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] role.driver = sql {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.168985 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.169113 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.169241 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.169369 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.169504 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.169644 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.169775 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.169904 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.170034 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.170162 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.170299 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.170466 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.170598 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.170726 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.170854 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.170983 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.171124 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.171254 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.171396 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.171525 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.171692 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.171828 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.171965 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.172095 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.172232 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.172362 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.172490 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.172618 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.172753 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.172882 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.173011 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.173139 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.173139 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.173139 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.173517 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.173647 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.173775 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.173903 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.174031 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.174166 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.174297 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.174453 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.174584 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.174712 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.174841 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.174990 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.175120 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.175249 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.175378 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.175507 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.175635 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.175771 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.175900 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.176028 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.176156 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.176283 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.176411 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.176547 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.176683 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.176833 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.176963 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.177091 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.177219 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] auth.external = None {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.177355 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.177485 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.177614 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] token.caching = True {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.177614 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.177856 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.177986 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.178122 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.178251 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.178429 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] auth.password = None {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.178577 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.178706 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] auth.token = None {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.178835 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.178974 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.179110 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.179238 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.179367 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.179496 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.179624 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.179759 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.179889 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.180017 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.180145 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.180273 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.180410 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.180548 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.180678 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.180806 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] credential.caching = True {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.180934 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.181061 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.181189 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.181329 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.181459 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.181597 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.181774 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.181911 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.182039 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.182177 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.182314 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] database.connection = **** {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.182466 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.182596 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.182725 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.183406 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.183573 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.183728 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.183859 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.183989 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.184117 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.184245 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.184373 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.184510 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.184651 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.184787 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.184787 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.185031 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.185160 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.185295 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.185425 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.185553 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.185686 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.185821 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] federation.caching = True {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.185950 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.186086 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.186216 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.186344 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.186522 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.186653 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.186782 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.186919 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.187049 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.187178 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.187316 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.187445 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.187573 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.187709 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.187839 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Nov 02 07:15:50.187969 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.187969 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.187969 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.187969 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.187969 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] identity.caching = True {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.187969 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.187969 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.187969 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.187969 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.189026 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.189026 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.189026 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.189026 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.189026 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.189026 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.189026 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.189026 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.189026 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.190149 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.190149 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.190149 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.190149 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.190149 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.190149 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.190149 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.190149 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.190149 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.191369 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.191369 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.191369 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.191369 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.191369 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.191369 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.191369 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.191369 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.191369 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.192475 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.192475 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.192475 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.192475 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.192475 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.192475 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.192475 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.192475 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.192475 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.193529 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.193529 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.193529 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.193529 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.193529 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.193529 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.193529 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.193529 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.193529 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.194596 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.194596 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] ldap.user = None {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.194596 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.194596 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.195086 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.195086 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.195086 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.195440 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.195566 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.195785 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.196026 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.196266 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.196507 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.196743 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.196991 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.197241 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.197490 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.197740 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.198103 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.198343 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.199613 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.199870 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.200117 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.200364 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.200613 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.200867 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.201122 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.201366 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.201625 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.201861 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.202111 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.202352 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.202619 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.202862 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.203114 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.203360 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.203597 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.203836 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.204082 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] resource.caching = True {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.204336 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.204586 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.204826 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.205076 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.205328 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.205567 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.205813 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.206053 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.206294 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.206569 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] role.caching = True {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.206819 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] role.driver = sql {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.207052 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.207304 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.207554 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.207806 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.208056 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.208304 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.208552 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.208801 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.209049 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.209282 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.209416 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81502) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:15:50.209548 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.209676 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81502) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:15:50.209814 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.209946 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81502) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:15:50.210077 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.210239 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.210676 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.210921 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.211177 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.211431 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.211684 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.211940 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.212179 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.212439 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.212697 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.212942 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.213180 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.213414 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.213650 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.213905 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.214666 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.214942 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.215193 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.215437 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.215674 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.215916 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] token.caching = True {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.216158 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.216410 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.216654 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.216913 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.217164 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.217405 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.217653 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.217899 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.218092 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81502) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:15:50.218241 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.218437 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.218627 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.218865 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.219113 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.219393 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.219627 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.219872 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.220138 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.220405 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] database.connection = **** {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.220651 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.220899 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.221141 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.221387 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.221628 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.221883 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.222107 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.222347 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.222617 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.222859 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.223106 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.223348 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.223599 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.223839 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.224087 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.224321 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.224565 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.224803 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:15:50.224976 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Nov 02 07:15:50.247559 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81502) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:15:50.247777 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81502) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:15:50.251559 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81501) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:15:50.251798 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81501) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:15:50.252025 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81501) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:15:50.260598 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81501) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:15:50.281800 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81501) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:15:50.282029 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81501) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:15:50.300624 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81502) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:15:50.319766 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81501) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:15:50.349394 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81502) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:15:50.349621 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81502) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:15:50.356945 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81501) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:15:50.357185 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81501) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:15:50.504773 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81502) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:15:50.505000 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81502) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:15:50.505198 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81502) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:15:50.505313 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81501) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:15:50.505397 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81502) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:15:50.505551 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81501) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:15:50.505634 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81502) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:15:50.505776 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81501) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:15:50.505858 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81502) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:15:50.505998 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81501) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:15:50.506220 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81501) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:15:50.506466 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81501) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:15:50.529231 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81502) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:15:50.529458 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81502) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:15:50.529678 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81502) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:15:50.529898 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81502) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:15:50.530113 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81502) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:15:50.530195 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81501) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:15:50.530323 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81502) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:15:50.530425 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81501) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:15:50.530562 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81502) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:15:50.530656 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81501) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:15:50.530826 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81501) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:15:50.531052 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81501) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:15:50.531273 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81501) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:15:50.531490 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81501) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:15:50.541438 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81502) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:15:50.542753 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81501) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:15:50.555044 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81501) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:15:50.555220 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81501) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:15:50.555454 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81501) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:15:50.555678 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81501) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:15:50.555900 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81501) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:15:50.556146 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81501) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:15:50.556366 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81501) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:15:50.556586 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81502) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:15:50.556814 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81502) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:15:50.557033 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81502) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:15:50.557248 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81502) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:15:50.557461 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81502) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:15:50.557671 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81502) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:15:50.558540 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81502) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:15:50.575860 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:15:50.575971 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:15:50.576078 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Nov 02 07:15:50.576162 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Nov 02 07:15:50.576418 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:15:50.576499 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:15:50.576616 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Nov 02 07:15:50.576700 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Nov 02 07:15:50.576886 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:15:50.576968 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:15:50.577143 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:15:50.577233 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:15:50.577399 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:15:50.577643 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:15:50.577722 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:15:50.577899 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:15:50.587236 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:15:50.587338 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:15:50.587512 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:15:50.587595 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:15:50.587780 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:15:50.587862 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:15:50.588057 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:15:50.588140 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:15:50.595056 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81502) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 07:15:50.595419 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81501) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 07:15:50.597926 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81502) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 07:15:50.598205 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:15:50.598315 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81501) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 07:15:50.598503 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:15:50.598630 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:15:50.598779 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:15:50.598919 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:15:50.599056 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:15:50.599208 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:15:50.599312 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:15:50.599496 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:15:50.599576 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:15:50.599764 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:15:50.599845 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:15:50.600041 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:15:50.600191 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:15:50.600321 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:15:50.600462 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:15:50.600597 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:15:50.600726 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:15:50.600874 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:15:50.601148 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:15:50.624563 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:15:50.626008 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81502) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 07:15:50.626287 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:15:50.627757 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:15:50.629222 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81501) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 07:15:50.629508 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:15:50.632627 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81502) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 07:15:50.632912 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:15:50.633190 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:15:50.634032 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81501) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 07:15:50.634322 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:15:50.634740 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:15:50.639723 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81502) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 07:15:50.640003 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:15:50.641580 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81501) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 07:15:50.641874 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:15:50.644173 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81502) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 07:15:50.644886 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:15:50.644886 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:15:50.645003 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:15:50.645246 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:15:50.645560 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:15:50.645782 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:15:50.648194 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:15:50.648293 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81501) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 07:15:50.648293 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:15:50.648293 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:15:50.648293 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:15:50.648293 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:15:50.648293 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:15:50.648611 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:15:50.648611 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:15:50.661639 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:15:50.661913 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:15:50.664710 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:15:50.664998 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:15:50.665768 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81502) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 07:15:50.669525 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:15:50.669810 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:15:50.669980 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81501) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 07:15:50.673110 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:15:50.673523 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:15:50.674012 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81502) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 07:15:50.677145 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81502) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 07:15:50.677762 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81501) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 07:15:50.680378 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:15:50.680677 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:15:50.680956 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81501) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 07:15:50.681046 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:15:50.681246 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:15:50.681528 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:15:50.681811 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:15:50.684571 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:15:50.684877 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:15:50.685170 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:15:50.685578 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:15:50.685869 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:15:50.686162 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:15:50.697592 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81502) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 07:15:50.697876 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:15:50.698141 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:15:50.698934 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:15:50.702929 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81501) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 07:15:50.703221 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:15:50.703490 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:15:50.703764 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:15:50.705777 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:15:50.707335 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:15:50.708860 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:15:50.709133 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:15:50.710573 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:15:50.711772 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81502) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 07:15:50.712052 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:15:50.712232 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:15:50.712434 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:15:50.712713 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:15:50.713002 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:15:50.713789 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:15:50.714243 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:15:50.717424 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81501) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 07:15:50.717716 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:15:50.718028 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:15:50.718392 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:15:50.718681 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:15:50.724882 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81502) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 07:15:50.725128 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:15:50.725502 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:15:50.725782 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:15:50.726071 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:15:50.726378 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:15:50.726653 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:15:50.731168 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81501) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 07:15:50.731464 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:15:50.731754 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:15:50.732044 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:15:50.732330 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:15:50.732610 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:15:50.732891 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:15:50.742829 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81502) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 07:15:50.745898 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81502) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 07:15:50.748838 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:15:50.748927 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81501) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 07:15:50.750267 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:15:50.751882 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81502) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 07:15:50.752015 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81501) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 07:15:50.752164 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:15:50.752439 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:15:50.754803 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:15:50.756249 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:15:50.757745 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81501) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 07:15:50.758034 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:15:50.758323 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:15:50.758785 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:15:50.760241 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81502) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 07:15:50.763869 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:15:50.764153 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:15:50.764432 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:15:50.764702 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:15:50.766142 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:15:50.767641 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81501) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 07:15:50.770511 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:15:50.770811 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:15:50.771097 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:15:50.771374 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:15:50.772460 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81502) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 07:15:50.772746 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:15:50.773024 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:15:50.780831 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81502) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 07:15:50.780940 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81501) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 07:15:50.781110 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:15:50.781194 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:15:50.781383 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:15:50.781477 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:15:50.781651 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:15:50.781917 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:15:50.782184 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:15:50.782478 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:15:50.782762 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:15:50.783026 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:15:50.783290 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:15:50.783565 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:15:50.783874 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:15:50.784145 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:15:50.784409 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:15:50.788282 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81501) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 07:15:50.788570 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:15:50.789179 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:15:50.789472 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:15:50.789750 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:15:50.790025 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:15:50.790300 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:15:50.790620 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:15:50.790906 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:15:50.791181 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:15:50.791456 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:15:50.791723 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:15:50.791992 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:15:50.793241 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:15:50.817577 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55f307e0d9c0 pid: 81502 (default app) Nov 02 07:15:50.820834 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf2aabd-91c5-42a8-bc3f-4dbd57ca27ce None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:50.821119 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf2aabd-91c5-42a8-bc3f-4dbd57ca27ce None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:50.821388 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf2aabd-91c5-42a8-bc3f-4dbd57ca27ce None None] PATH_INFO: `/v3` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:50.826208 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1/1] 10.222.0.18 () {50 vars in 791 bytes} [Wed Nov 2 07:15:50 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:50.827597 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55f307e0d9c0 pid: 81501 (default app) Nov 02 07:15:57.568550 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e23101-9a7b-4bda-9cc3-5abcf490f147 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:57.569083 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e23101-9a7b-4bda-9cc3-5abcf490f147 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:57.569083 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e23101-9a7b-4bda-9cc3-5abcf490f147 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:57.570384 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 2/2] 10.222.0.18 () {54 vars in 927 bytes} [Wed Nov 2 07:15:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Nov 02 07:15:57.578874 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba5b373-ce04-4214-b38e-e6d0d647fef4 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:57.579325 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba5b373-ce04-4214-b38e-e6d0d647fef4 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:57.579457 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba5b373-ce04-4214-b38e-e6d0d647fef4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:57.649610 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: INFO dbcounter [None req-fba5b373-ce04-4214-b38e-e6d0d647fef4 None None] Registered counter for database keystone Nov 02 07:15:57.669138 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG oslo_db.sqlalchemy.engines [None req-fba5b373-ce04-4214-b38e-e6d0d647fef4 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81501) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:326}} Nov 02 07:15:57.675390 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writer thread running {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Nov 02 07:15:57.855169 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG passlib.handlers.bcrypt [None req-fba5b373-ce04-4214-b38e-e6d0d647fef4 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81501) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Nov 02 07:15:57.855738 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG passlib.handlers.bcrypt [None req-fba5b373-ce04-4214-b38e-e6d0d647fef4 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81501) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Nov 02 07:15:57.912722 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-fba5b373-ce04-4214-b38e-e6d0d647fef4 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:15:57.914294 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-fba5b373-ce04-4214-b38e-e6d0d647fef4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:15:58.040842 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1/3] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Nov 2 07:15:57 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 465 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Nov 02 07:15:58.474285 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e23101-9a7b-4bda-9cc3-5abcf490f147 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:58.543245 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: INFO dbcounter [None req-26e23101-9a7b-4bda-9cc3-5abcf490f147 None None] Registered counter for database keystone Nov 02 07:15:58.564587 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG oslo_db.sqlalchemy.engines [None req-26e23101-9a7b-4bda-9cc3-5abcf490f147 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81502) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:326}} Nov 02 07:15:58.570719 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writer thread running {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Nov 02 07:15:58.690699 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e23101-9a7b-4bda-9cc3-5abcf490f147 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:58.694879 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc3ae08-3c21-4d86-9992-a0580fafbb62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:58.695501 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc3ae08-3c21-4d86-9992-a0580fafbb62 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:58.695778 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc3ae08-3c21-4d86-9992-a0580fafbb62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:58.696044 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc3ae08-3c21-4d86-9992-a0580fafbb62 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:58.704793 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc3ae08-3c21-4d86-9992-a0580fafbb62 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:58.756243 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: WARNING keystone.common.rbac_enforcer.enforcer [None req-3fc3ae08-3c21-4d86-9992-a0580fafbb62 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Nov 02 07:15:58.756675 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc3ae08-3c21-4d86-9992-a0580fafbb62 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:58.766296 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 3/4] 10.222.0.18 () {58 vars in 1324 bytes} [Wed Nov 2 07:15:58 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Nov 02 07:15:58.772188 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba5b373-ce04-4214-b38e-e6d0d647fef4 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:58.789196 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba5b373-ce04-4214-b38e-e6d0d647fef4 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:58.793710 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463f81c2-2b3d-43b4-834e-3232f995003f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:58.794855 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463f81c2-2b3d-43b4-834e-3232f995003f admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:58.795137 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463f81c2-2b3d-43b4-834e-3232f995003f admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:58.795407 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463f81c2-2b3d-43b4-834e-3232f995003f admin admin] PATH_INFO: `/v3/projects/4ad924d40efc4e06bdd880c155ca36a4` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:58.797007 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463f81c2-2b3d-43b4-834e-3232f995003f admin admin] RBAC: Authorizing `identity:get_project(project_id=4ad924d40efc4e06bdd880c155ca36a4)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:58.847260 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: WARNING keystone.common.rbac_enforcer.enforcer [None req-463f81c2-2b3d-43b4-834e-3232f995003f admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Nov 02 07:15:58.847601 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463f81c2-2b3d-43b4-834e-3232f995003f admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:58.852022 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 2/5] 10.222.0.18 () {56 vars in 1207 bytes} [Wed Nov 2 07:15:58 2022] GET /identity/v3/projects/4ad924d40efc4e06bdd880c155ca36a4 => generated 344 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:16:00.971712 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8a711b-5831-4dcf-aa71-999c73fa7172 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:00.972243 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8a711b-5831-4dcf-aa71-999c73fa7172 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:00.972243 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8a711b-5831-4dcf-aa71-999c73fa7172 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:00.976267 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 4/6] 10.222.0.18 () {54 vars in 927 bytes} [Wed Nov 2 07:16:00 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:16:00.982580 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce2d13e-aea2-427f-8466-af49ede17cdd None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:00.982998 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce2d13e-aea2-427f-8466-af49ede17cdd None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:00.983124 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce2d13e-aea2-427f-8466-af49ede17cdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:01.002475 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-8ce2d13e-aea2-427f-8466-af49ede17cdd None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:16:01.003962 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-8ce2d13e-aea2-427f-8466-af49ede17cdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:16:01.011745 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 3/7] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Nov 2 07:16:00 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Nov 02 07:16:01.275053 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8a711b-5831-4dcf-aa71-999c73fa7172 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:01.286887 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8a711b-5831-4dcf-aa71-999c73fa7172 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:01.290993 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bd5b9c-df4c-4c61-96a0-3d679276ca2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:01.292116 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bd5b9c-df4c-4c61-96a0-3d679276ca2e admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:01.292394 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bd5b9c-df4c-4c61-96a0-3d679276ca2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:01.292665 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bd5b9c-df4c-4c61-96a0-3d679276ca2e admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:01.300063 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bd5b9c-df4c-4c61-96a0-3d679276ca2e admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:01.301814 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bd5b9c-df4c-4c61-96a0-3d679276ca2e admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:01.311069 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 5/8] 10.222.0.18 () {58 vars in 1324 bytes} [Wed Nov 2 07:16:01 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Nov 02 07:16:01.316177 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce2d13e-aea2-427f-8466-af49ede17cdd None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:01.325249 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce2d13e-aea2-427f-8466-af49ede17cdd None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:01.329551 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088ba84c-c3ce-4c46-8c94-ceb458a5427c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:01.330173 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088ba84c-c3ce-4c46-8c94-ceb458a5427c admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:01.330493 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088ba84c-c3ce-4c46-8c94-ceb458a5427c admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:01.330777 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088ba84c-c3ce-4c46-8c94-ceb458a5427c admin admin] PATH_INFO: `/v3/users/b87b731cb4ea4070ae4dad0c3c393570` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:01.332074 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088ba84c-c3ce-4c46-8c94-ceb458a5427c admin admin] RBAC: Authorizing `identity:get_user(user_id=b87b731cb4ea4070ae4dad0c3c393570)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:01.333880 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088ba84c-c3ce-4c46-8c94-ceb458a5427c admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:01.335414 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 4/9] 10.222.0.18 () {56 vars in 1198 bytes} [Wed Nov 2 07:16:01 2022] GET /identity/v3/users/b87b731cb4ea4070ae4dad0c3c393570 => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:03.523677 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d25a515-612b-4689-85cb-2b79ca78f816 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:03.524202 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d25a515-612b-4689-85cb-2b79ca78f816 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:03.524202 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d25a515-612b-4689-85cb-2b79ca78f816 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:03.525683 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 6/10] 10.222.0.18 () {54 vars in 927 bytes} [Wed Nov 2 07:16:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Nov 02 07:16:03.531027 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fced6e0-2d85-4c8e-948a-f86111d91bfc None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:03.531438 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fced6e0-2d85-4c8e-948a-f86111d91bfc None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:03.531644 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fced6e0-2d85-4c8e-948a-f86111d91bfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:03.550058 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-0fced6e0-2d85-4c8e-948a-f86111d91bfc None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:16:03.551266 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-0fced6e0-2d85-4c8e-948a-f86111d91bfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:16:03.569396 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0de4a69-22f5-46dc-887d-8b02c236ea03 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:03.569683 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0de4a69-22f5-46dc-887d-8b02c236ea03 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:03.569959 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0de4a69-22f5-46dc-887d-8b02c236ea03 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:03.571294 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 7/11] 10.222.0.18 () {54 vars in 927 bytes} [Wed Nov 2 07:16:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:16:03.576590 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc21a178-7d0f-4fdc-9f6b-2392863dcef2 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:03.576901 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc21a178-7d0f-4fdc-9f6b-2392863dcef2 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:03.577230 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc21a178-7d0f-4fdc-9f6b-2392863dcef2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:03.618742 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG passlib.handlers.bcrypt [None req-bc21a178-7d0f-4fdc-9f6b-2392863dcef2 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81502) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Nov 02 07:16:03.619716 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG passlib.handlers.bcrypt [None req-bc21a178-7d0f-4fdc-9f6b-2392863dcef2 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81502) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Nov 02 07:16:03.633755 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 5/12] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Nov 2 07:16:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:16:03.668642 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-bc21a178-7d0f-4fdc-9f6b-2392863dcef2 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:16:03.670227 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-bc21a178-7d0f-4fdc-9f6b-2392863dcef2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:16:03.767262 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 8/13] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Nov 2 07:16:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Nov 02 07:16:03.901684 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fced6e0-2d85-4c8e-948a-f86111d91bfc None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:03.925569 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:04.003965 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fced6e0-2d85-4c8e-948a-f86111d91bfc None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:04.035737 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc21a178-7d0f-4fdc-9f6b-2392863dcef2 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:04.036504 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf160f0-d0e3-4555-b511-98aede20a459 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:04.037140 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf160f0-d0e3-4555-b511-98aede20a459 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:04.037420 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf160f0-d0e3-4555-b511-98aede20a459 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:04.037844 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf160f0-d0e3-4555-b511-98aede20a459 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:04.048984 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf160f0-d0e3-4555-b511-98aede20a459 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:16:04.058184 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-3bf160f0-d0e3-4555-b511-98aede20a459 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 07:16:04.058184 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:04.058184 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:16:04.058184 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:16:04.058184 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:16:04.058184 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:16:04.058184 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:16:04.058184 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:16:04.058184 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:16:04.058184 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:16:04.058184 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:16:04.058184 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:16:04.058184 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:16:04.058184 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:16:04.058184 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:16:04.058184 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:16:04.059142 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:16:04.059142 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:16:04.059142 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:16:04.059142 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:16:04.059142 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:16:04.059142 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:16:04.059142 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:16:04.059142 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:16:04.059142 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:16:04.059142 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:16:04.059142 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:16:04.059142 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Nov 02 07:16:04.059142 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:16:04.059142 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:16:04.059142 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:16:04.059142 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:16:04.059946 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:16:04.059946 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:16:04.059946 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:16:04.059946 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:16:04.059946 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:16:04.059946 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:16:04.059946 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:16:04.059946 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:16:04.059946 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:16:04.059946 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:16:04.059946 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 07:16:04.059946 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Nov 02 07:16:04.059946 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 6/14] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Nov 2 07:16:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:16:04.064924 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf160f0-d0e3-4555-b511-98aede20a459 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:04.133989 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc21a178-7d0f-4fdc-9f6b-2392863dcef2 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:04.136825 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:04.143831 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:04.162643 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf160f0-d0e3-4555-b511-98aede20a459 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:04.174946 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17c5dd0-657b-4ec3-85b0-7149160f4ec1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:04.175820 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17c5dd0-657b-4ec3-85b0-7149160f4ec1 None admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:04.176100 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17c5dd0-657b-4ec3-85b0-7149160f4ec1 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:04.176374 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17c5dd0-657b-4ec3-85b0-7149160f4ec1 None admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:04.177321 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17c5dd0-657b-4ec3-85b0-7149160f4ec1 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:04.178798 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17c5dd0-657b-4ec3-85b0-7149160f4ec1 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:04.189906 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.sql.core [None req-f17c5dd0-657b-4ec3-85b0-7149160f4ec1 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Nov 02 07:16:04.189906 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Nov 02 07:16:04.189906 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [parameters: {'id': '62704288916d43c6b9f81301ad4d52e5', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Nov 02 07:16:04.189906 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81502) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Nov 02 07:16:04.195704 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12074da9-4d8b-48a3-a0d2-2ae98a75a986 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:04.196313 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12074da9-4d8b-48a3-a0d2-2ae98a75a986 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:04.196595 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12074da9-4d8b-48a3-a0d2-2ae98a75a986 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:04.196867 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12074da9-4d8b-48a3-a0d2-2ae98a75a986 None admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:04.198228 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12074da9-4d8b-48a3-a0d2-2ae98a75a986 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:04.198568 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12074da9-4d8b-48a3-a0d2-2ae98a75a986 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:04.199953 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12074da9-4d8b-48a3-a0d2-2ae98a75a986 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:04.208556 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-f17c5dd0-657b-4ec3-85b0-7149160f4ec1 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Nov 02 07:16:04.208556 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:04.208556 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Nov 02 07:16:04.208556 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application self.dialect.do_execute( Nov 02 07:16:04.208556 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Nov 02 07:16:04.208556 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Nov 02 07:16:04.208556 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Nov 02 07:16:04.208556 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application result = self._query(query) Nov 02 07:16:04.208556 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Nov 02 07:16:04.208556 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application conn.query(q) Nov 02 07:16:04.208556 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Nov 02 07:16:04.208556 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Nov 02 07:16:04.208556 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Nov 02 07:16:04.208556 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application result.read() Nov 02 07:16:04.209316 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Nov 02 07:16:04.209316 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Nov 02 07:16:04.209316 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Nov 02 07:16:04.209316 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application packet.raise_for_error() Nov 02 07:16:04.209316 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Nov 02 07:16:04.209316 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Nov 02 07:16:04.209316 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Nov 02 07:16:04.209316 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Nov 02 07:16:04.209316 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Nov 02 07:16:04.209316 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:16:04.209316 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Nov 02 07:16:04.209316 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:16:04.209316 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:04.209316 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Nov 02 07:16:04.209316 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return method(*args, **kwargs) Nov 02 07:16:04.209316 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Nov 02 07:16:04.209316 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return project_ref.to_dict() Nov 02 07:16:04.209316 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Nov 02 07:16:04.210246 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application next(self.gen) Nov 02 07:16:04.210246 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1068, in _transaction_scope Nov 02 07:16:04.210246 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application yield resource Nov 02 07:16:04.210246 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Nov 02 07:16:04.210246 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application next(self.gen) Nov 02 07:16:04.210246 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 670, in _session Nov 02 07:16:04.210246 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application self.session.rollback() Nov 02 07:16:04.210246 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Nov 02 07:16:04.210246 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application self.force_reraise() Nov 02 07:16:04.210246 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Nov 02 07:16:04.210246 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise self.value Nov 02 07:16:04.210246 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 667, in _session Nov 02 07:16:04.210246 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Nov 02 07:16:04.210246 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 695, in _end_session_transaction Nov 02 07:16:04.210246 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application session.commit() Nov 02 07:16:04.210246 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Nov 02 07:16:04.210246 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Nov 02 07:16:04.211126 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Nov 02 07:16:04.211126 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application self._prepare_impl() Nov 02 07:16:04.211126 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Nov 02 07:16:04.211126 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application self.session.flush() Nov 02 07:16:04.211126 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Nov 02 07:16:04.211126 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application self._flush(objects) Nov 02 07:16:04.211126 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Nov 02 07:16:04.211126 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Nov 02 07:16:04.211126 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Nov 02 07:16:04.211126 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application compat.raise_( Nov 02 07:16:04.211126 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Nov 02 07:16:04.211126 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception Nov 02 07:16:04.211126 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Nov 02 07:16:04.211126 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application flush_context.execute() Nov 02 07:16:04.211126 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Nov 02 07:16:04.211126 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rec.execute(self) Nov 02 07:16:04.211126 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Nov 02 07:16:04.212999 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Nov 02 07:16:04.212999 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Nov 02 07:16:04.212999 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application _emit_insert_statements( Nov 02 07:16:04.212999 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Nov 02 07:16:04.212999 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application c = connection._execute_20( Nov 02 07:16:04.212999 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Nov 02 07:16:04.212999 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Nov 02 07:16:04.212999 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Nov 02 07:16:04.212999 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Nov 02 07:16:04.212999 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Nov 02 07:16:04.212999 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ret = self._execute_context( Nov 02 07:16:04.212999 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Nov 02 07:16:04.212999 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Nov 02 07:16:04.212999 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Nov 02 07:16:04.212999 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Nov 02 07:16:04.213832 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Nov 02 07:16:04.213832 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception Nov 02 07:16:04.213832 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Nov 02 07:16:04.213832 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application self.dialect.do_execute( Nov 02 07:16:04.213832 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Nov 02 07:16:04.213832 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Nov 02 07:16:04.213832 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Nov 02 07:16:04.213832 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application result = self._query(query) Nov 02 07:16:04.213832 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Nov 02 07:16:04.213832 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application conn.query(q) Nov 02 07:16:04.213832 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Nov 02 07:16:04.213832 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Nov 02 07:16:04.213832 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Nov 02 07:16:04.213832 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application result.read() Nov 02 07:16:04.213832 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Nov 02 07:16:04.213832 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Nov 02 07:16:04.213832 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Nov 02 07:16:04.214651 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application packet.raise_for_error() Nov 02 07:16:04.214651 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Nov 02 07:16:04.214651 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Nov 02 07:16:04.214651 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Nov 02 07:16:04.214651 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Nov 02 07:16:04.214651 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Nov 02 07:16:04.214651 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Nov 02 07:16:04.214651 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [parameters: {'id': '62704288916d43c6b9f81301ad4d52e5', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Nov 02 07:16:04.214651 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Nov 02 07:16:04.214651 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:16:04.214651 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 07:16:04.214651 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:16:04.214651 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:04.214651 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Nov 02 07:16:04.214651 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Nov 02 07:16:04.215369 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Nov 02 07:16:04.215369 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Nov 02 07:16:04.215369 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name Default. Nov 02 07:16:04.215369 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:16:04.215369 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 07:16:04.215369 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:16:04.215369 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:04.215369 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:16:04.215369 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:16:04.215369 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:16:04.215369 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:16:04.215369 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:16:04.215369 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:16:04.215369 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:16:04.215369 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:16:04.215369 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:16:04.215369 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:16:04.216233 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 128, in post Nov 02 07:16:04.216233 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_domain( Nov 02 07:16:04.216233 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:16:04.216233 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:16:04.216233 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 765, in create_domain Nov 02 07:16:04.216233 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application is_domain_project = self.create_project( Nov 02 07:16:04.216233 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:16:04.216233 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:16:04.216233 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Nov 02 07:16:04.216233 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.Conflict( Nov 02 07:16:04.216233 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Nov 02 07:16:04.216233 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:16:04.216233 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 9/15] 10.222.0.18 () {60 vars in 1137 bytes} [Wed Nov 2 07:16:04 2022] POST /identity/v3/domains => generated 190 bytes in 174 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Nov 02 07:16:04.216845 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 7/16] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Nov 2 07:16:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:16:04.217825 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17c5dd0-657b-4ec3-85b0-7149160f4ec1 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:04.217941 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12074da9-4d8b-48a3-a0d2-2ae98a75a986 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:04.317362 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17c5dd0-657b-4ec3-85b0-7149160f4ec1 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:04.317988 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12074da9-4d8b-48a3-a0d2-2ae98a75a986 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:04.352084 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b838d4-a293-45a9-a430-1da32be3e28d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:04.352919 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b838d4-a293-45a9-a430-1da32be3e28d None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:04.353193 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b838d4-a293-45a9-a430-1da32be3e28d None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:04.353444 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d39b022-06bd-4cd7-a57c-39446277c8ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:04.353566 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b838d4-a293-45a9-a430-1da32be3e28d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:04.354068 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d39b022-06bd-4cd7-a57c-39446277c8ed None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:04.354347 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d39b022-06bd-4cd7-a57c-39446277c8ed None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:04.354641 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d39b022-06bd-4cd7-a57c-39446277c8ed None admin] PATH_INFO: `/v3/users/b87b731cb4ea4070ae4dad0c3c393570` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:04.356031 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d39b022-06bd-4cd7-a57c-39446277c8ed None admin] RBAC: Authorizing `identity:get_user(user_id=b87b731cb4ea4070ae4dad0c3c393570)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:04.357475 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d39b022-06bd-4cd7-a57c-39446277c8ed None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:04.359061 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 8/17] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Nov 2 07:16:04 2022] GET /identity/v3/users/b87b731cb4ea4070ae4dad0c3c393570 => generated 247 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:16:04.364451 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d39b022-06bd-4cd7-a57c-39446277c8ed None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:04.372240 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b838d4-a293-45a9-a430-1da32be3e28d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:04.373695 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b838d4-a293-45a9-a430-1da32be3e28d None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:04.378418 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:04.390270 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-20b838d4-a293-45a9-a430-1da32be3e28d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:16:04.390270 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:04.390270 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:16:04.390270 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:16:04.390270 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:16:04.390270 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:16:04.390270 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:16:04.390270 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:16:04.390270 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:16:04.390270 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:16:04.390270 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:16:04.390270 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:16:04.390270 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:16:04.390270 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:16:04.390270 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:16:04.390270 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:16:04.391351 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:16:04.391351 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:16:04.391351 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:16:04.391351 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:16:04.391351 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:16:04.391351 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:16:04.391351 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:16:04.391351 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:16:04.391351 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:16:04.391351 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:16:04.391351 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:16:04.391351 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:16:04.391351 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:16:04.391351 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:16:04.391351 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:16:04.391351 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:16:04.391351 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:16:04.392270 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:16:04.392270 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:16:04.392270 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:16:04.392270 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 10/18] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Nov 2 07:16:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 07:16:04.397188 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b838d4-a293-45a9-a430-1da32be3e28d None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:04.421983 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:04.466023 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d39b022-06bd-4cd7-a57c-39446277c8ed None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:04.497989 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b838d4-a293-45a9-a430-1da32be3e28d None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:04.499631 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2584865-eecb-4b18-9741-9574c5d5d2af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:04.500265 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2584865-eecb-4b18-9741-9574c5d5d2af None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:04.500547 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2584865-eecb-4b18-9741-9574c5d5d2af None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:04.500817 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2584865-eecb-4b18-9741-9574c5d5d2af None admin] PATH_INFO: `/v3/domains/default` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:04.501918 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2584865-eecb-4b18-9741-9574c5d5d2af None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:04.503385 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2584865-eecb-4b18-9741-9574c5d5d2af None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:04.504917 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 9/19] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Nov 2 07:16:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:16:04.511189 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2584865-eecb-4b18-9741-9574c5d5d2af None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:04.531777 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec2517f-05cf-4762-b155-d8ddc6a48f4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:04.532400 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec2517f-05cf-4762-b155-d8ddc6a48f4a None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:04.532678 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec2517f-05cf-4762-b155-d8ddc6a48f4a None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:04.532958 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec2517f-05cf-4762-b155-d8ddc6a48f4a None admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:04.533604 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec2517f-05cf-4762-b155-d8ddc6a48f4a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:04.533906 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec2517f-05cf-4762-b155-d8ddc6a48f4a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:04.535316 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec2517f-05cf-4762-b155-d8ddc6a48f4a None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:04.553665 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 11/20] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Nov 2 07:16:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:16:04.595474 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:04.616822 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2584865-eecb-4b18-9741-9574c5d5d2af None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:04.659025 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632deaea-5afe-4f8d-bc43-a871c5367bde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:04.659709 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632deaea-5afe-4f8d-bc43-a871c5367bde None admin] REQUEST_METHOD: `PUT` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:04.659991 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632deaea-5afe-4f8d-bc43-a871c5367bde None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:04.660275 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632deaea-5afe-4f8d-bc43-a871c5367bde None admin] PATH_INFO: `/v3/domains/default/users/b87b731cb4ea4070ae4dad0c3c393570/roles/c4bf019fc6c44c71a6c984f6d0bab75f` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:04.662257 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632deaea-5afe-4f8d-bc43-a871c5367bde None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=b87b731cb4ea4070ae4dad0c3c393570, role_id=c4bf019fc6c44c71a6c984f6d0bab75f)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:04.663779 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632deaea-5afe-4f8d-bc43-a871c5367bde None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:04.780149 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 10/21] 10.222.0.18 () {58 vars in 1361 bytes} [Wed Nov 2 07:16:04 2022] PUT /identity/v3/domains/default/users/b87b731cb4ea4070ae4dad0c3c393570/roles/c4bf019fc6c44c71a6c984f6d0bab75f => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 07:16:07.231596 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14373fc-d40c-4d88-83fd-9cc1c1ee2653 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:07.232643 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14373fc-d40c-4d88-83fd-9cc1c1ee2653 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:07.233125 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14373fc-d40c-4d88-83fd-9cc1c1ee2653 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:07.234787 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 12/22] 10.222.0.18 () {54 vars in 927 bytes} [Wed Nov 2 07:16:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:16:07.241494 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc8f86d-29b4-4295-9fc6-7181c546225c None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:07.242455 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc8f86d-29b4-4295-9fc6-7181c546225c None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:07.242904 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc8f86d-29b4-4295-9fc6-7181c546225c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:07.268069 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-7fc8f86d-29b4-4295-9fc6-7181c546225c None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:16:07.269880 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-7fc8f86d-29b4-4295-9fc6-7181c546225c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:16:07.363126 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 11/23] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Nov 2 07:16:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:16:07.373738 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae199c8-08c0-40d7-8c4c-58e4bf0658a7 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:07.374025 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae199c8-08c0-40d7-8c4c-58e4bf0658a7 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:07.374307 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae199c8-08c0-40d7-8c4c-58e4bf0658a7 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:07.375873 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 13/24] 10.222.0.18 () {54 vars in 927 bytes} [Wed Nov 2 07:16:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:16:07.381270 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9faac4-b945-4973-a834-a308a7388c9f None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:07.381561 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9faac4-b945-4973-a834-a308a7388c9f None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:07.381840 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9faac4-b945-4973-a834-a308a7388c9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:07.400708 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-4e9faac4-b945-4973-a834-a308a7388c9f None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:16:07.401891 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-4e9faac4-b945-4973-a834-a308a7388c9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:16:07.467943 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:07.472976 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 12/25] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Nov 2 07:16:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Nov 02 07:16:07.563703 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a2049f-b6c3-4b3b-8831-a49ee16f6dc0 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:07.564334 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a2049f-b6c3-4b3b-8831-a49ee16f6dc0 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:07.564796 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a2049f-b6c3-4b3b-8831-a49ee16f6dc0 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:07.566606 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 14/26] 10.222.0.18 () {54 vars in 927 bytes} [Wed Nov 2 07:16:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:16:07.571703 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec13c200-6d9f-49c6-bad2-063f8474fe02 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:07.572298 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec13c200-6d9f-49c6-bad2-063f8474fe02 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:07.572851 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec13c200-6d9f-49c6-bad2-063f8474fe02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:07.597206 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-ec13c200-6d9f-49c6-bad2-063f8474fe02 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:16:07.598737 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-ec13c200-6d9f-49c6-bad2-063f8474fe02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:16:07.686293 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 13/27] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Nov 2 07:16:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:16:07.786286 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a2049f-b6c3-4b3b-8831-a49ee16f6dc0 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:07.880225 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:07.902597 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a2049f-b6c3-4b3b-8831-a49ee16f6dc0 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:07.939643 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0066d6-1d84-47db-ad5f-d6c0645b5ba4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:07.940596 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0066d6-1d84-47db-ad5f-d6c0645b5ba4 None admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:07.941064 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0066d6-1d84-47db-ad5f-d6c0645b5ba4 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:07.941522 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0066d6-1d84-47db-ad5f-d6c0645b5ba4 None admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:07.942402 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0066d6-1d84-47db-ad5f-d6c0645b5ba4 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:07.943982 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0066d6-1d84-47db-ad5f-d6c0645b5ba4 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:07.998924 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699279a6-6046-48ec-b733-bc2d9a817b19 None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:07.999534 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699279a6-6046-48ec-b733-bc2d9a817b19 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:08.000035 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699279a6-6046-48ec-b733-bc2d9a817b19 None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:08.004344 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 14/28] 10.222.0.18 () {54 vars in 927 bytes} [Wed Nov 2 07:16:07 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Nov 02 07:16:08.010180 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4832a6e3-0241-4860-a300-1a94483ec9b3 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:08.010734 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4832a6e3-0241-4860-a300-1a94483ec9b3 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:08.011198 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4832a6e3-0241-4860-a300-1a94483ec9b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:08.050560 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-4832a6e3-0241-4860-a300-1a94483ec9b3 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:16:08.052039 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-4832a6e3-0241-4860-a300-1a94483ec9b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:16:08.121941 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 15/29] 10.222.0.18 () {60 vars in 1131 bytes} [Wed Nov 2 07:16:07 2022] POST /identity/v3/roles => generated 219 bytes in 336 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:16:08.123239 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0066d6-1d84-47db-ad5f-d6c0645b5ba4 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:08.145404 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 15/30] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Nov 2 07:16:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Nov 02 07:16:08.146638 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4832a6e3-0241-4860-a300-1a94483ec9b3 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:08.224843 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:08.275630 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4832a6e3-0241-4860-a300-1a94483ec9b3 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:08.319849 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735668cc-a89e-4ed5-9719-438a3e4fccd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:08.321135 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735668cc-a89e-4ed5-9719-438a3e4fccd3 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:08.321612 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735668cc-a89e-4ed5-9719-438a3e4fccd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:08.323149 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735668cc-a89e-4ed5-9719-438a3e4fccd3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:08.334193 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0066d6-1d84-47db-ad5f-d6c0645b5ba4 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:08.339353 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735668cc-a89e-4ed5-9719-438a3e4fccd3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:16:08.345044 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-735668cc-a89e-4ed5-9719-438a3e4fccd3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 07:16:08.345044 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:08.345044 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:16:08.345044 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:16:08.345044 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:16:08.345044 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:16:08.345044 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:16:08.345044 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:16:08.345044 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:16:08.345044 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:16:08.345044 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:16:08.345044 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:16:08.345044 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:16:08.345044 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:16:08.345044 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:16:08.345044 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:16:08.345933 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:16:08.345933 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:16:08.345933 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:16:08.345933 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:16:08.345933 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:16:08.345933 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:16:08.345933 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:16:08.345933 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:16:08.345933 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:16:08.345933 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:16:08.345933 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:16:08.345933 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Nov 02 07:16:08.345933 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:16:08.345933 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:16:08.345933 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:16:08.345933 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:16:08.346796 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:16:08.346796 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:16:08.346796 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:16:08.346796 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:16:08.346796 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:16:08.346796 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:16:08.346796 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:16:08.346796 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:16:08.346796 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:16:08.346796 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:16:08.346796 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 07:16:08.346796 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Nov 02 07:16:08.346796 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 16/31] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Nov 2 07:16:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:16:08.348226 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ae118b-062b-4104-a123-43796a0ad1ba None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:08.348226 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ae118b-062b-4104-a123-43796a0ad1ba None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:08.348457 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ae118b-062b-4104-a123-43796a0ad1ba None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:08.353873 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 17/32] 10.222.0.18 () {54 vars in 927 bytes} [Wed Nov 2 07:16:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:16:08.356516 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ae118b-062b-4104-a123-43796a0ad1ba None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:08.383851 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbc24c2-d3a6-4b27-af21-7ab0e0cde82e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:08.385579 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbc24c2-d3a6-4b27-af21-7ab0e0cde82e None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:08.386126 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbc24c2-d3a6-4b27-af21-7ab0e0cde82e None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:08.386665 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbc24c2-d3a6-4b27-af21-7ab0e0cde82e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:08.397542 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:08.400515 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbc24c2-d3a6-4b27-af21-7ab0e0cde82e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:08.402717 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbc24c2-d3a6-4b27-af21-7ab0e0cde82e None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:08.420757 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-5dbc24c2-d3a6-4b27-af21-7ab0e0cde82e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:16:08.420757 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:08.420757 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:16:08.420757 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:16:08.420757 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:16:08.420757 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:16:08.420757 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:16:08.420757 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:16:08.420757 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:16:08.420757 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:16:08.420757 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:16:08.420757 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:16:08.420757 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:16:08.420757 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:16:08.420757 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:16:08.420757 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:16:08.421662 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:16:08.421662 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:16:08.421662 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:16:08.421662 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:16:08.421662 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:16:08.421662 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:16:08.421662 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:16:08.421662 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:16:08.421662 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:16:08.421662 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:16:08.421662 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:16:08.421662 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:16:08.421662 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:16:08.421662 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:16:08.421662 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:16:08.421662 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:16:08.421662 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:16:08.422477 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:16:08.422477 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:16:08.422477 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:16:08.424729 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 16/33] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Nov 2 07:16:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:16:08.426411 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a6bc72-607d-40ec-9a2e-f204b9758a04 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:08.426948 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a6bc72-607d-40ec-9a2e-f204b9758a04 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:08.427414 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a6bc72-607d-40ec-9a2e-f204b9758a04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:08.448874 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-d0a6bc72-607d-40ec-9a2e-f204b9758a04 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:16:08.451938 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-d0a6bc72-607d-40ec-9a2e-f204b9758a04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:16:08.490750 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ae118b-062b-4104-a123-43796a0ad1ba None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:08.520543 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:08.533839 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 17/34] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Nov 2 07:16:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:16:08.534239 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415c6b28-cc65-4819-9507-2a060060442a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:08.534995 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415c6b28-cc65-4819-9507-2a060060442a None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:08.535376 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415c6b28-cc65-4819-9507-2a060060442a None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:08.535502 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a6bc72-607d-40ec-9a2e-f204b9758a04 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:08.535818 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415c6b28-cc65-4819-9507-2a060060442a None admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:08.536593 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415c6b28-cc65-4819-9507-2a060060442a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:08.536992 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415c6b28-cc65-4819-9507-2a060060442a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:08.538701 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415c6b28-cc65-4819-9507-2a060060442a None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:08.550589 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 18/35] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Nov 2 07:16:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:16:08.551692 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415c6b28-cc65-4819-9507-2a060060442a None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:08.663775 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a6bc72-607d-40ec-9a2e-f204b9758a04 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:08.672206 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415c6b28-cc65-4819-9507-2a060060442a None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:08.691785 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:08.708464 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b971b5d2-e6f6-4695-a05a-9b05d25214b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:08.709326 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b971b5d2-e6f6-4695-a05a-9b05d25214b4 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:08.709614 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b971b5d2-e6f6-4695-a05a-9b05d25214b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:08.709891 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b971b5d2-e6f6-4695-a05a-9b05d25214b4 None admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:08.710569 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b971b5d2-e6f6-4695-a05a-9b05d25214b4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:08.710875 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b971b5d2-e6f6-4695-a05a-9b05d25214b4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:08.711553 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cb5bad-7c3f-4922-8cf8-941a9b027d4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:08.712260 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b971b5d2-e6f6-4695-a05a-9b05d25214b4 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:08.712737 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cb5bad-7c3f-4922-8cf8-941a9b027d4b None admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:08.713265 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cb5bad-7c3f-4922-8cf8-941a9b027d4b None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:08.713736 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cb5bad-7c3f-4922-8cf8-941a9b027d4b None admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:08.714579 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cb5bad-7c3f-4922-8cf8-941a9b027d4b None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:08.716131 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cb5bad-7c3f-4922-8cf8-941a9b027d4b None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:08.725526 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 18/36] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Nov 2 07:16:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:08.727725 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b971b5d2-e6f6-4695-a05a-9b05d25214b4 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:08.739422 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 19/37] 10.222.0.18 () {60 vars in 1131 bytes} [Wed Nov 2 07:16:08 2022] POST /identity/v3/roles => generated 225 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:16:08.740234 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cb5bad-7c3f-4922-8cf8-941a9b027d4b None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:08.842005 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:08.938714 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b971b5d2-e6f6-4695-a05a-9b05d25214b4 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:08.979547 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd62c564-2a6d-4cc3-898b-34f812154bf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:08.980573 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd62c564-2a6d-4cc3-898b-34f812154bf2 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:08.980858 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd62c564-2a6d-4cc3-898b-34f812154bf2 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:08.981128 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd62c564-2a6d-4cc3-898b-34f812154bf2 None admin] PATH_INFO: `/v3/users/b87b731cb4ea4070ae4dad0c3c393570` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:08.982561 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd62c564-2a6d-4cc3-898b-34f812154bf2 None admin] RBAC: Authorizing `identity:get_user(user_id=b87b731cb4ea4070ae4dad0c3c393570)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:08.984022 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd62c564-2a6d-4cc3-898b-34f812154bf2 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:08.985991 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 19/38] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Nov 2 07:16:08 2022] GET /identity/v3/users/b87b731cb4ea4070ae4dad0c3c393570 => generated 247 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:08.986844 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd62c564-2a6d-4cc3-898b-34f812154bf2 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:09.020609 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cb5bad-7c3f-4922-8cf8-941a9b027d4b None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:09.059046 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:09.066847 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f8f83a-8780-4306-9a8f-884acaac3d32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:09.067524 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f8f83a-8780-4306-9a8f-884acaac3d32 None admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:09.067759 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f8f83a-8780-4306-9a8f-884acaac3d32 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:09.068045 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f8f83a-8780-4306-9a8f-884acaac3d32 None admin] PATH_INFO: `/v3/projects` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:09.068705 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f8f83a-8780-4306-9a8f-884acaac3d32 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:09.070167 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f8f83a-8780-4306-9a8f-884acaac3d32 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:09.102021 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 20/39] 10.222.0.18 () {60 vars in 1141 bytes} [Wed Nov 2 07:16:08 2022] POST /identity/v3/projects => generated 301 bytes in 362 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:16:09.102773 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f8f83a-8780-4306-9a8f-884acaac3d32 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:09.121595 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd62c564-2a6d-4cc3-898b-34f812154bf2 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:09.177239 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a876aff3-7ef1-405f-be20-79fe3798ac01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:09.179107 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a876aff3-7ef1-405f-be20-79fe3798ac01 None admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:09.179629 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a876aff3-7ef1-405f-be20-79fe3798ac01 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:09.180366 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a876aff3-7ef1-405f-be20-79fe3798ac01 None admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:09.182311 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a876aff3-7ef1-405f-be20-79fe3798ac01 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:09.184319 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a876aff3-7ef1-405f-be20-79fe3798ac01 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:09.208879 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 20/40] 10.222.0.18 () {60 vars in 1131 bytes} [Wed Nov 2 07:16:08 2022] POST /identity/v3/roles => generated 223 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:16:09.223206 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:09.233938 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f8f83a-8780-4306-9a8f-884acaac3d32 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:09.278609 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3a99c6-3dca-4899-ad33-666b133d30d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:09.279428 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3a99c6-3dca-4899-ad33-666b133d30d6 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:09.279639 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3a99c6-3dca-4899-ad33-666b133d30d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:09.279893 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3a99c6-3dca-4899-ad33-666b133d30d6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:09.281197 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3a99c6-3dca-4899-ad33-666b133d30d6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:09.282809 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3a99c6-3dca-4899-ad33-666b133d30d6 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:09.284879 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 21/41] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Nov 2 07:16:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:09.291230 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a876aff3-7ef1-405f-be20-79fe3798ac01 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:09.412573 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a876aff3-7ef1-405f-be20-79fe3798ac01 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:09.424615 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:09.427776 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c91ce9-cab9-43c0-b7f4-6c3e62e0e712 None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:09.428144 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c91ce9-cab9-43c0-b7f4-6c3e62e0e712 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:09.428419 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c91ce9-cab9-43c0-b7f4-6c3e62e0e712 None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:09.429769 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 22/42] 10.222.0.18 () {54 vars in 927 bytes} [Wed Nov 2 07:16:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Nov 02 07:16:09.435311 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779c4b8f-aa0c-49b3-ad5f-d077b33bd660 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:09.435597 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779c4b8f-aa0c-49b3-ad5f-d077b33bd660 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:09.435873 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779c4b8f-aa0c-49b3-ad5f-d077b33bd660 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:09.454084 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca9b078-36b2-4e86-83f3-d27e567cc7a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:09.454713 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca9b078-36b2-4e86-83f3-d27e567cc7a5 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:09.454997 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca9b078-36b2-4e86-83f3-d27e567cc7a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:09.455277 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca9b078-36b2-4e86-83f3-d27e567cc7a5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:09.456007 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca9b078-36b2-4e86-83f3-d27e567cc7a5 None admin] RBAC: Adding query filter params (role.id=c4bf019fc6c44c71a6c984f6d0bab75f, scope.domain.id=default, user.id=b87b731cb4ea4070ae4dad0c3c393570) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:09.456313 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca9b078-36b2-4e86-83f3-d27e567cc7a5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:09.457769 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca9b078-36b2-4e86-83f3-d27e567cc7a5 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:09.459211 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-779c4b8f-aa0c-49b3-ad5f-d077b33bd660 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:16:09.460508 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-779c4b8f-aa0c-49b3-ad5f-d077b33bd660 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:16:09.468098 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 21/43] 10.222.0.18 () {56 vars in 1322 bytes} [Wed Nov 2 07:16:09 2022] GET /identity/v3/role_assignments?user.id=b87b731cb4ea4070ae4dad0c3c393570&scope.domain.id=default&role.id=c4bf019fc6c44c71a6c984f6d0bab75f => generated 530 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:16:09.537429 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 23/44] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Nov 2 07:16:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Nov 02 07:16:09.807247 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca9b078-36b2-4e86-83f3-d27e567cc7a5 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:09.902713 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca9b078-36b2-4e86-83f3-d27e567cc7a5 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:09.934761 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad16bac4-b041-430e-97e3-28620f2e52dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:09.935382 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad16bac4-b041-430e-97e3-28620f2e52dd None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:09.935662 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad16bac4-b041-430e-97e3-28620f2e52dd None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:09.935933 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad16bac4-b041-430e-97e3-28620f2e52dd None admin] PATH_INFO: `/v3/domains/default` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:09.937009 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad16bac4-b041-430e-97e3-28620f2e52dd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:09.938464 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad16bac4-b041-430e-97e3-28620f2e52dd None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:09.939981 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 22/45] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Nov 2 07:16:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:09.945395 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779c4b8f-aa0c-49b3-ad5f-d077b33bd660 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:10.032484 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:10.041191 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779c4b8f-aa0c-49b3-ad5f-d077b33bd660 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:10.074145 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f26b097-2a6c-47be-aa78-c9169da422d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:10.074702 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f26b097-2a6c-47be-aa78-c9169da422d6 None admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:10.074983 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f26b097-2a6c-47be-aa78-c9169da422d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:10.075253 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f26b097-2a6c-47be-aa78-c9169da422d6 None admin] PATH_INFO: `/v3/projects` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:10.075897 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f26b097-2a6c-47be-aa78-c9169da422d6 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:10.077333 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f26b097-2a6c-47be-aa78-c9169da422d6 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:10.102056 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 24/46] 10.222.0.18 () {60 vars in 1141 bytes} [Wed Nov 2 07:16:09 2022] POST /identity/v3/projects => generated 312 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 07:16:12.227629 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe0417c-f452-4d7b-9256-fdaeb83831d0 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:12.228200 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe0417c-f452-4d7b-9256-fdaeb83831d0 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:12.228291 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe0417c-f452-4d7b-9256-fdaeb83831d0 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:12.229804 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 23/47] 10.222.0.18 () {54 vars in 927 bytes} [Wed Nov 2 07:16:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:16:12.234971 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03863aad-379e-4e22-b2fd-c9aaa4abdc30 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:12.235357 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03863aad-379e-4e22-b2fd-c9aaa4abdc30 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:12.235523 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03863aad-379e-4e22-b2fd-c9aaa4abdc30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:12.253728 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-03863aad-379e-4e22-b2fd-c9aaa4abdc30 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:16:12.254934 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-03863aad-379e-4e22-b2fd-c9aaa4abdc30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:16:12.318961 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 25/48] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Nov 2 07:16:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Nov 02 07:16:12.583502 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe0417c-f452-4d7b-9256-fdaeb83831d0 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:12.631440 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:12.688312 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe0417c-f452-4d7b-9256-fdaeb83831d0 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:12.720891 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509eea30-5a21-434c-b501-d6039b1a2ecf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:12.721510 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509eea30-5a21-434c-b501-d6039b1a2ecf None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:12.721789 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509eea30-5a21-434c-b501-d6039b1a2ecf None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:12.722059 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509eea30-5a21-434c-b501-d6039b1a2ecf None admin] PATH_INFO: `/v3/domains/default` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:12.723182 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509eea30-5a21-434c-b501-d6039b1a2ecf None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:12.724600 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509eea30-5a21-434c-b501-d6039b1a2ecf None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:12.726143 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 24/49] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Nov 2 07:16:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:12.732827 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03863aad-379e-4e22-b2fd-c9aaa4abdc30 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:12.803466 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:12.828052 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03863aad-379e-4e22-b2fd-c9aaa4abdc30 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:12.861607 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3287a7f-119a-403f-89d0-1db120419c8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:12.862143 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3287a7f-119a-403f-89d0-1db120419c8f None admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:12.862451 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3287a7f-119a-403f-89d0-1db120419c8f None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:12.862728 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3287a7f-119a-403f-89d0-1db120419c8f None admin] PATH_INFO: `/v3/projects` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:12.863370 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3287a7f-119a-403f-89d0-1db120419c8f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:12.864800 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3287a7f-119a-403f-89d0-1db120419c8f None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:12.889145 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 26/50] 10.222.0.18 () {60 vars in 1140 bytes} [Wed Nov 2 07:16:12 2022] POST /identity/v3/projects => generated 298 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 07:16:15.027132 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bdcc06-2ede-4334-a426-1cc2ed898cef None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:15.027636 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bdcc06-2ede-4334-a426-1cc2ed898cef None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:15.027741 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bdcc06-2ede-4334-a426-1cc2ed898cef None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:15.029003 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 25/51] 10.222.0.18 () {54 vars in 927 bytes} [Wed Nov 2 07:16:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:16:15.034258 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9e335e-893a-43b8-be55-fbc04be1129e None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:15.034680 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9e335e-893a-43b8-be55-fbc04be1129e None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:15.034853 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9e335e-893a-43b8-be55-fbc04be1129e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:15.054419 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-5d9e335e-893a-43b8-be55-fbc04be1129e None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:16:15.056153 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-5d9e335e-893a-43b8-be55-fbc04be1129e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:16:15.120014 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 27/52] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Nov 2 07:16:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:16:15.389989 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bdcc06-2ede-4334-a426-1cc2ed898cef None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:15.488108 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bdcc06-2ede-4334-a426-1cc2ed898cef None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:15.498114 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:15.522350 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9b0b0a-8496-4751-8b28-8012951caab3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:15.523020 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9b0b0a-8496-4751-8b28-8012951caab3 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:15.523301 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9b0b0a-8496-4751-8b28-8012951caab3 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:15.523572 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9b0b0a-8496-4751-8b28-8012951caab3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:15.525343 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9b0b0a-8496-4751-8b28-8012951caab3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:15.526800 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9b0b0a-8496-4751-8b28-8012951caab3 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:15.528330 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 26/53] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Nov 2 07:16:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:15.533755 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9e335e-893a-43b8-be55-fbc04be1129e None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:15.586206 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:15.628386 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9e335e-893a-43b8-be55-fbc04be1129e None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:15.661176 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0065d250-db29-40f5-a656-e41ad2ddcfec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:15.661782 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0065d250-db29-40f5-a656-e41ad2ddcfec None admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:15.662062 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0065d250-db29-40f5-a656-e41ad2ddcfec None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:15.662331 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0065d250-db29-40f5-a656-e41ad2ddcfec None admin] PATH_INFO: `/v3/users` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:15.663001 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0065d250-db29-40f5-a656-e41ad2ddcfec None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:15.664428 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0065d250-db29-40f5-a656-e41ad2ddcfec None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:15.691737 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 28/54] 10.222.0.18 () {60 vars in 1132 bytes} [Wed Nov 2 07:16:15 2022] POST /identity/v3/users => generated 275 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:16:19.031098 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1c2e61-b1c2-4958-a235-465d7a0808d8 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:19.032119 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1c2e61-b1c2-4958-a235-465d7a0808d8 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:19.032629 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1c2e61-b1c2-4958-a235-465d7a0808d8 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:19.034147 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 27/55] 10.222.0.18 () {54 vars in 927 bytes} [Wed Nov 2 07:16:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:16:19.039725 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871db1bf-d376-409b-acc2-ba1d178da23b None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:19.040641 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871db1bf-d376-409b-acc2-ba1d178da23b None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:19.041127 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871db1bf-d376-409b-acc2-ba1d178da23b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:19.068457 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d571db8d-df15-4abe-aa8f-ae97ffde60e4 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:19.069055 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d571db8d-df15-4abe-aa8f-ae97ffde60e4 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:19.069513 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d571db8d-df15-4abe-aa8f-ae97ffde60e4 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:19.071110 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 28/56] 10.222.0.18 () {54 vars in 927 bytes} [Wed Nov 2 07:16:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Nov 02 07:16:19.073445 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-871db1bf-d376-409b-acc2-ba1d178da23b None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:16:19.074871 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-871db1bf-d376-409b-acc2-ba1d178da23b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:16:19.076419 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa4a4f7-493f-4543-a425-4a827a5a2802 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:19.076839 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa4a4f7-493f-4543-a425-4a827a5a2802 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:19.077258 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa4a4f7-493f-4543-a425-4a827a5a2802 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:19.107826 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-6fa4a4f7-493f-4543-a425-4a827a5a2802 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:16:19.111938 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-6fa4a4f7-493f-4543-a425-4a827a5a2802 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:16:19.210008 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 29/57] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Nov 2 07:16:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:16:19.241189 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 29/58] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Nov 2 07:16:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:16:19.311879 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9bcb62-a76c-42b7-bf19-6e4a1722a934 None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:19.312402 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9bcb62-a76c-42b7-bf19-6e4a1722a934 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:19.312812 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9bcb62-a76c-42b7-bf19-6e4a1722a934 None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:19.314317 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 30/59] 10.222.0.18 () {54 vars in 927 bytes} [Wed Nov 2 07:16:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:16:19.319790 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c71091-6307-45c3-b4cb-feafb3220001 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:19.320231 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c71091-6307-45c3-b4cb-feafb3220001 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:19.320665 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c71091-6307-45c3-b4cb-feafb3220001 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:19.343283 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-32c71091-6307-45c3-b4cb-feafb3220001 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:16:19.344733 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-32c71091-6307-45c3-b4cb-feafb3220001 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:16:19.417026 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 30/60] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Nov 2 07:16:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:16:19.478252 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da24d867-e6f0-46e0-ba64-ce1b3ca1c2e5 None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:19.478811 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da24d867-e6f0-46e0-ba64-ce1b3ca1c2e5 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:19.479475 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da24d867-e6f0-46e0-ba64-ce1b3ca1c2e5 None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:19.480959 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 31/61] 10.222.0.18 () {54 vars in 927 bytes} [Wed Nov 2 07:16:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:16:19.486259 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0cf74e-726e-44cd-b9bb-aaaa62dd0733 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:19.486720 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0cf74e-726e-44cd-b9bb-aaaa62dd0733 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:19.487127 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0cf74e-726e-44cd-b9bb-aaaa62dd0733 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:19.511290 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-be0cf74e-726e-44cd-b9bb-aaaa62dd0733 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:16:19.512654 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-be0cf74e-726e-44cd-b9bb-aaaa62dd0733 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:16:19.555159 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:19.560413 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da24d867-e6f0-46e0-ba64-ce1b3ca1c2e5 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:19.605929 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 31/62] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Nov 2 07:16:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:16:19.614390 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0cf74e-726e-44cd-b9bb-aaaa62dd0733 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:19.626765 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:19.752722 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da24d867-e6f0-46e0-ba64-ce1b3ca1c2e5 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:19.793864 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b904b002-5040-45e6-87d4-5de639c9da7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:19.794527 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b904b002-5040-45e6-87d4-5de639c9da7a None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:19.794813 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b904b002-5040-45e6-87d4-5de639c9da7a None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:19.795086 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b904b002-5040-45e6-87d4-5de639c9da7a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:19.809195 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b904b002-5040-45e6-87d4-5de639c9da7a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:16:19.814066 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-b904b002-5040-45e6-87d4-5de639c9da7a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 07:16:19.814066 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:19.814066 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:16:19.814066 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:16:19.814066 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:16:19.814066 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:16:19.814066 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:16:19.814066 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:16:19.814066 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:16:19.814066 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:16:19.814066 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:16:19.814066 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:16:19.814066 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:16:19.814066 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:16:19.814066 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:16:19.814066 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:16:19.814927 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:16:19.814927 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:16:19.814927 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:16:19.814927 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:16:19.814927 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:16:19.814927 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:16:19.814927 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:16:19.814927 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:16:19.814927 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:16:19.814927 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:16:19.814927 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:16:19.814927 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Nov 02 07:16:19.814927 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:16:19.814927 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:16:19.814927 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:16:19.814927 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:16:19.815665 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:16:19.815665 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:16:19.815665 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:16:19.815665 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:16:19.815665 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:16:19.815665 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:16:19.815665 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:16:19.815665 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:16:19.815665 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:16:19.815665 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:16:19.815665 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 07:16:19.815665 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Nov 02 07:16:19.815665 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 32/63] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Nov 2 07:16:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 07:16:19.816978 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a600bff5-68d3-4a7e-961b-992e12c9c4e8 None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:19.817214 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a600bff5-68d3-4a7e-961b-992e12c9c4e8 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:19.817491 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a600bff5-68d3-4a7e-961b-992e12c9c4e8 None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:19.818873 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 33/64] 10.222.0.18 () {54 vars in 927 bytes} [Wed Nov 2 07:16:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:16:19.819429 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a600bff5-68d3-4a7e-961b-992e12c9c4e8 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:19.837803 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0cf74e-726e-44cd-b9bb-aaaa62dd0733 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:19.875115 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49faa8c-58b0-4359-b997-6517a11ae723 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:19.875676 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49faa8c-58b0-4359-b997-6517a11ae723 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:19.875955 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49faa8c-58b0-4359-b997-6517a11ae723 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:19.876227 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49faa8c-58b0-4359-b997-6517a11ae723 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:19.887717 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49faa8c-58b0-4359-b997-6517a11ae723 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:16:19.894146 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-b49faa8c-58b0-4359-b997-6517a11ae723 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Nov 02 07:16:19.894146 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:19.894146 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:16:19.894146 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:16:19.894146 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:16:19.894146 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:16:19.894146 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:16:19.894146 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:16:19.894146 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:16:19.894146 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:16:19.894146 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:16:19.894146 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:16:19.894146 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:16:19.894146 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:16:19.894146 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:16:19.894146 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:16:19.895023 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:16:19.895023 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:16:19.895023 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:16:19.895023 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:16:19.895023 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:16:19.895023 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:16:19.895023 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:16:19.895023 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:16:19.895023 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:16:19.895023 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:16:19.895023 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:16:19.895023 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:16:19.895023 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:16:19.895023 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:16:19.895023 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:16:19.895023 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:16:19.895858 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:16:19.895858 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:16:19.895858 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:16:19.895858 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:16:19.895858 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:16:19.895858 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:16:19.895858 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:16:19.895858 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:16:19.895858 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:16:19.895858 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:16:19.895858 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Nov 02 07:16:19.895858 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:16:19.895858 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 32/65] 10.222.0.18 () {56 vars in 1114 bytes} [Wed Nov 2 07:16:19 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:16:19.896495 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49faa8c-58b0-4359-b997-6517a11ae723 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:19.925184 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:19.931338 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a600bff5-68d3-4a7e-961b-992e12c9c4e8 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:19.932798 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:19.965802 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10962e68-45d9-42bd-8fbd-1a4b58d0cc86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:19.966460 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10962e68-45d9-42bd-8fbd-1a4b58d0cc86 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:19.966740 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10962e68-45d9-42bd-8fbd-1a4b58d0cc86 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:19.967017 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10962e68-45d9-42bd-8fbd-1a4b58d0cc86 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:19.976723 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10962e68-45d9-42bd-8fbd-1a4b58d0cc86 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:16:19.982427 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-10962e68-45d9-42bd-8fbd-1a4b58d0cc86 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Nov 02 07:16:19.982427 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:19.982427 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:16:19.982427 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:16:19.982427 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:16:19.982427 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:16:19.982427 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:16:19.982427 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:16:19.982427 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:16:19.982427 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:16:19.982427 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:16:19.982427 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:16:19.982427 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:16:19.982427 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:16:19.982427 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:16:19.982427 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:16:19.983328 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:16:19.983328 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:16:19.983328 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:16:19.983328 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:16:19.983328 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:16:19.983328 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:16:19.983328 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:16:19.983328 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:16:19.983328 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:16:19.983328 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:16:19.983328 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:16:19.983328 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Nov 02 07:16:19.983328 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:16:19.983328 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:16:19.983328 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:16:19.983328 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:16:19.984067 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:16:19.984067 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:16:19.984067 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:16:19.984067 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:16:19.984067 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:16:19.984067 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:16:19.984067 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:16:19.984067 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:16:19.984067 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:16:19.984067 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:16:19.984067 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Nov 02 07:16:19.984067 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Nov 02 07:16:19.984067 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 34/66] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Nov 2 07:16:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:16:19.984806 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b8dbe0-3b73-4541-a107-1b6d449c9de9 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:19.985127 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b8dbe0-3b73-4541-a107-1b6d449c9de9 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:19.985331 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b8dbe0-3b73-4541-a107-1b6d449c9de9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:20.001903 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49faa8c-58b0-4359-b997-6517a11ae723 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:20.002985 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-52b8dbe0-3b73-4541-a107-1b6d449c9de9 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:16:20.004178 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-52b8dbe0-3b73-4541-a107-1b6d449c9de9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:16:20.035056 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fd4dd2-b876-4e3b-bd94-bf884305151a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:20.035672 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fd4dd2-b876-4e3b-bd94-bf884305151a None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:20.035956 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fd4dd2-b876-4e3b-bd94-bf884305151a None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:20.036230 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fd4dd2-b876-4e3b-bd94-bf884305151a None admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:20.036897 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fd4dd2-b876-4e3b-bd94-bf884305151a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:20.037198 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fd4dd2-b876-4e3b-bd94-bf884305151a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:20.038618 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fd4dd2-b876-4e3b-bd94-bf884305151a None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:20.051389 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 33/67] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Nov 2 07:16:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:20.052199 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fd4dd2-b876-4e3b-bd94-bf884305151a None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:20.071455 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 35/68] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Nov 2 07:16:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:16:20.072486 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b8dbe0-3b73-4541-a107-1b6d449c9de9 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:20.148870 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:20.160368 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fd4dd2-b876-4e3b-bd94-bf884305151a None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:20.172436 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:20.183459 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b8dbe0-3b73-4541-a107-1b6d449c9de9 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:20.197307 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731d02c8-c23b-4b29-af7c-109243d0bd4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:20.197925 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731d02c8-c23b-4b29-af7c-109243d0bd4f None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:20.198203 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731d02c8-c23b-4b29-af7c-109243d0bd4f None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:20.198507 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731d02c8-c23b-4b29-af7c-109243d0bd4f None admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:20.199315 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731d02c8-c23b-4b29-af7c-109243d0bd4f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:20.199613 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731d02c8-c23b-4b29-af7c-109243d0bd4f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:20.201115 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731d02c8-c23b-4b29-af7c-109243d0bd4f None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:20.209112 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 34/69] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Nov 2 07:16:20 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:20.210077 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731d02c8-c23b-4b29-af7c-109243d0bd4f None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:20.223089 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13375d26-059a-46f1-a0a4-f85509b44bbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:20.223723 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13375d26-059a-46f1-a0a4-f85509b44bbe None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:20.224005 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13375d26-059a-46f1-a0a4-f85509b44bbe None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:20.224277 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13375d26-059a-46f1-a0a4-f85509b44bbe None admin] PATH_INFO: `/v3/roles/member` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:20.236060 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13375d26-059a-46f1-a0a4-f85509b44bbe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:16:20.240937 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-13375d26-059a-46f1-a0a4-f85509b44bbe None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Nov 02 07:16:20.240937 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:20.240937 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:16:20.240937 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:16:20.240937 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:16:20.240937 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:16:20.240937 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:16:20.240937 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:16:20.240937 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:16:20.240937 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:16:20.240937 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:16:20.240937 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:16:20.240937 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:16:20.240937 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:16:20.240937 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:16:20.240937 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:16:20.241752 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:16:20.241752 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:16:20.241752 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:16:20.241752 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:16:20.241752 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:16:20.241752 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:16:20.241752 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:16:20.241752 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:16:20.241752 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:16:20.241752 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:16:20.241752 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:16:20.241752 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Nov 02 07:16:20.241752 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:16:20.241752 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:16:20.241752 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:16:20.241752 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:16:20.243095 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:16:20.243095 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:16:20.243095 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:16:20.243095 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:16:20.243095 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:16:20.243095 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:16:20.243095 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:16:20.243095 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:16:20.243095 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:16:20.243095 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:16:20.243095 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Nov 02 07:16:20.243095 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Nov 02 07:16:20.243095 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 36/70] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Nov 2 07:16:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:16:20.247206 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13375d26-059a-46f1-a0a4-f85509b44bbe None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:20.323904 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731d02c8-c23b-4b29-af7c-109243d0bd4f None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:20.349257 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13375d26-059a-46f1-a0a4-f85509b44bbe None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:20.358548 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b66d90e-c0da-43ce-8b30-39a6d4737259 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:20.359159 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b66d90e-c0da-43ce-8b30-39a6d4737259 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:20.359463 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b66d90e-c0da-43ce-8b30-39a6d4737259 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:20.359737 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b66d90e-c0da-43ce-8b30-39a6d4737259 None admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:20.360386 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b66d90e-c0da-43ce-8b30-39a6d4737259 None admin] RBAC: Adding query filter params (name=member) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:20.360687 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b66d90e-c0da-43ce-8b30-39a6d4737259 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:20.362076 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b66d90e-c0da-43ce-8b30-39a6d4737259 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:20.370404 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 35/71] 10.222.0.18 () {56 vars in 1101 bytes} [Wed Nov 2 07:16:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:20.371199 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b66d90e-c0da-43ce-8b30-39a6d4737259 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:20.386565 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:20.387729 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6861673c-2238-447b-8ddf-cdefb9166caa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:20.388395 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6861673c-2238-447b-8ddf-cdefb9166caa None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:20.388676 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6861673c-2238-447b-8ddf-cdefb9166caa None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:20.388977 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6861673c-2238-447b-8ddf-cdefb9166caa None admin] PATH_INFO: `/v3/users/b87b731cb4ea4070ae4dad0c3c393570` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:20.390099 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6861673c-2238-447b-8ddf-cdefb9166caa None admin] RBAC: Authorizing `identity:get_user(user_id=b87b731cb4ea4070ae4dad0c3c393570)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:20.391604 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6861673c-2238-447b-8ddf-cdefb9166caa None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:20.393168 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 37/72] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Nov 2 07:16:20 2022] GET /identity/v3/users/b87b731cb4ea4070ae4dad0c3c393570 => generated 247 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:20.393918 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6861673c-2238-447b-8ddf-cdefb9166caa None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:20.415462 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:20.477800 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b66d90e-c0da-43ce-8b30-39a6d4737259 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:20.493099 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6861673c-2238-447b-8ddf-cdefb9166caa None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:20.511969 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9088ff7-aed2-48fd-9273-49f2640a09fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:20.512602 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9088ff7-aed2-48fd-9273-49f2640a09fa None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:20.512894 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9088ff7-aed2-48fd-9273-49f2640a09fa None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:20.513199 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9088ff7-aed2-48fd-9273-49f2640a09fa None admin] PATH_INFO: `/v3/users/7e7f81d50d1a49b3a0076a19ded69866` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:20.526322 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058ff182-9c2b-4db1-bb8d-d6de44a31d6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:20.526958 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058ff182-9c2b-4db1-bb8d-d6de44a31d6b None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:20.527234 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058ff182-9c2b-4db1-bb8d-d6de44a31d6b None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:20.527503 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058ff182-9c2b-4db1-bb8d-d6de44a31d6b None admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:20.527739 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9088ff7-aed2-48fd-9273-49f2640a09fa None admin] RBAC: Authorizing `identity:get_user(user_id=7e7f81d50d1a49b3a0076a19ded69866)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:20.528153 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058ff182-9c2b-4db1-bb8d-d6de44a31d6b None admin] RBAC: Adding query filter params (name=member) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:20.528450 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058ff182-9c2b-4db1-bb8d-d6de44a31d6b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:20.529187 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9088ff7-aed2-48fd-9273-49f2640a09fa None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:20.529811 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058ff182-9c2b-4db1-bb8d-d6de44a31d6b None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:20.530843 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 36/73] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Nov 2 07:16:20 2022] GET /identity/v3/users/7e7f81d50d1a49b3a0076a19ded69866 => generated 275 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:20.531607 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9088ff7-aed2-48fd-9273-49f2640a09fa None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:20.538626 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 38/74] 10.222.0.18 () {56 vars in 1101 bytes} [Wed Nov 2 07:16:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:20.539441 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058ff182-9c2b-4db1-bb8d-d6de44a31d6b None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:20.604561 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:20.619425 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:20.634951 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9088ff7-aed2-48fd-9273-49f2640a09fa None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:20.637826 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058ff182-9c2b-4db1-bb8d-d6de44a31d6b None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:20.675112 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99330cc6-8e80-4e76-b5fd-9bc30f333663 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:20.675408 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174cefed-eee2-4e17-bb5f-a7223bf0f0de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:20.675684 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99330cc6-8e80-4e76-b5fd-9bc30f333663 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:20.675968 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99330cc6-8e80-4e76-b5fd-9bc30f333663 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:20.676052 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174cefed-eee2-4e17-bb5f-a7223bf0f0de None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:20.676244 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99330cc6-8e80-4e76-b5fd-9bc30f333663 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:20.676328 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174cefed-eee2-4e17-bb5f-a7223bf0f0de None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:20.676571 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174cefed-eee2-4e17-bb5f-a7223bf0f0de None admin] PATH_INFO: `/v3/users/7e7f81d50d1a49b3a0076a19ded69866` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:20.677427 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99330cc6-8e80-4e76-b5fd-9bc30f333663 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:20.677692 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174cefed-eee2-4e17-bb5f-a7223bf0f0de None admin] RBAC: Authorizing `identity:get_user(user_id=7e7f81d50d1a49b3a0076a19ded69866)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:20.678951 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99330cc6-8e80-4e76-b5fd-9bc30f333663 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:20.679160 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174cefed-eee2-4e17-bb5f-a7223bf0f0de None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:20.680761 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 39/75] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Nov 2 07:16:20 2022] GET /identity/v3/users/7e7f81d50d1a49b3a0076a19ded69866 => generated 275 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:20.685527 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 37/76] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Nov 2 07:16:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:20.685660 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174cefed-eee2-4e17-bb5f-a7223bf0f0de None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:20.685747 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99330cc6-8e80-4e76-b5fd-9bc30f333663 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:20.783925 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174cefed-eee2-4e17-bb5f-a7223bf0f0de None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:20.784922 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99330cc6-8e80-4e76-b5fd-9bc30f333663 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:20.817541 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4980f66-7712-4a69-9449-981b08dcac91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:20.817852 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec7de45-2f8e-4dca-ac90-72c587e1d111 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:20.818164 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4980f66-7712-4a69-9449-981b08dcac91 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:20.818471 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4980f66-7712-4a69-9449-981b08dcac91 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:20.818565 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec7de45-2f8e-4dca-ac90-72c587e1d111 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:20.818743 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4980f66-7712-4a69-9449-981b08dcac91 None admin] PATH_INFO: `/v3/projects/a6b063398180486c85fc258b5a513046` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:20.818829 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec7de45-2f8e-4dca-ac90-72c587e1d111 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:20.819048 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec7de45-2f8e-4dca-ac90-72c587e1d111 None admin] PATH_INFO: `/v3/projects/a6b063398180486c85fc258b5a513046` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:20.819845 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4980f66-7712-4a69-9449-981b08dcac91 None admin] RBAC: Authorizing `identity:get_project(project_id=a6b063398180486c85fc258b5a513046)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:20.820274 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec7de45-2f8e-4dca-ac90-72c587e1d111 None admin] RBAC: Authorizing `identity:get_project(project_id=a6b063398180486c85fc258b5a513046)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:20.821984 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4980f66-7712-4a69-9449-981b08dcac91 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:20.822442 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec7de45-2f8e-4dca-ac90-72c587e1d111 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:20.823648 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 40/77] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Nov 2 07:16:20 2022] GET /identity/v3/projects/a6b063398180486c85fc258b5a513046 => generated 298 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:20.828642 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 38/78] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Nov 2 07:16:20 2022] GET /identity/v3/projects/a6b063398180486c85fc258b5a513046 => generated 298 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:20.828769 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4980f66-7712-4a69-9449-981b08dcac91 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:20.828842 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec7de45-2f8e-4dca-ac90-72c587e1d111 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:20.835412 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:20.846775 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:20.924629 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec7de45-2f8e-4dca-ac90-72c587e1d111 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:20.928420 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4980f66-7712-4a69-9449-981b08dcac91 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:20.957287 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c673cbbf-e59d-4e35-a49c-932400b7d2e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:20.957908 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c673cbbf-e59d-4e35-a49c-932400b7d2e2 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:20.958196 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c673cbbf-e59d-4e35-a49c-932400b7d2e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:20.958500 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c673cbbf-e59d-4e35-a49c-932400b7d2e2 None admin] PATH_INFO: `/v3/projects/a6b063398180486c85fc258b5a513046` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:20.959610 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c673cbbf-e59d-4e35-a49c-932400b7d2e2 None admin] RBAC: Authorizing `identity:get_project(project_id=a6b063398180486c85fc258b5a513046)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:20.961081 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c673cbbf-e59d-4e35-a49c-932400b7d2e2 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:20.961952 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27131e5c-82cf-4a65-915f-8aa4b720fd08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:20.962575 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27131e5c-82cf-4a65-915f-8aa4b720fd08 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:20.962854 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27131e5c-82cf-4a65-915f-8aa4b720fd08 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:20.963120 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27131e5c-82cf-4a65-915f-8aa4b720fd08 None admin] PATH_INFO: `/v3/users/7e7f81d50d1a49b3a0076a19ded69866` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:20.963478 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 39/79] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Nov 2 07:16:20 2022] GET /identity/v3/projects/a6b063398180486c85fc258b5a513046 => generated 298 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:20.964226 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c673cbbf-e59d-4e35-a49c-932400b7d2e2 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:20.964322 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27131e5c-82cf-4a65-915f-8aa4b720fd08 None admin] RBAC: Authorizing `identity:get_user(user_id=7e7f81d50d1a49b3a0076a19ded69866)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:20.965682 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27131e5c-82cf-4a65-915f-8aa4b720fd08 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:20.971597 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 41/80] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Nov 2 07:16:20 2022] GET /identity/v3/users/7e7f81d50d1a49b3a0076a19ded69866 => generated 275 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:20.971597 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27131e5c-82cf-4a65-915f-8aa4b720fd08 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:21.043138 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:21.050688 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:21.065997 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27131e5c-82cf-4a65-915f-8aa4b720fd08 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:21.066962 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c673cbbf-e59d-4e35-a49c-932400b7d2e2 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:21.099319 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311b5823-b4bb-4d93-a2ba-fae2f4f22068 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:21.099970 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311b5823-b4bb-4d93-a2ba-fae2f4f22068 None admin] REQUEST_METHOD: `PUT` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:21.100254 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311b5823-b4bb-4d93-a2ba-fae2f4f22068 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:21.100528 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311b5823-b4bb-4d93-a2ba-fae2f4f22068 None admin] PATH_INFO: `/v3/projects/a6b063398180486c85fc258b5a513046/users/b87b731cb4ea4070ae4dad0c3c393570/roles/c4bf019fc6c44c71a6c984f6d0bab75f` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:21.100761 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca62d5a-c4ab-49d8-b176-a804ddf7a714 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:21.101361 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca62d5a-c4ab-49d8-b176-a804ddf7a714 None admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:21.101667 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca62d5a-c4ab-49d8-b176-a804ddf7a714 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:21.101988 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca62d5a-c4ab-49d8-b176-a804ddf7a714 None admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:21.103131 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca62d5a-c4ab-49d8-b176-a804ddf7a714 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:21.103259 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311b5823-b4bb-4d93-a2ba-fae2f4f22068 None admin] RBAC: Authorizing `identity:create_grant(project_id=a6b063398180486c85fc258b5a513046, user_id=b87b731cb4ea4070ae4dad0c3c393570, role_id=c4bf019fc6c44c71a6c984f6d0bab75f)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:21.104826 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca62d5a-c4ab-49d8-b176-a804ddf7a714 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:21.104944 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311b5823-b4bb-4d93-a2ba-fae2f4f22068 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:21.117300 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 42/81] 10.222.0.18 () {58 vars in 1439 bytes} [Wed Nov 2 07:16:20 2022] PUT /identity/v3/projects/a6b063398180486c85fc258b5a513046/users/b87b731cb4ea4070ae4dad0c3c393570/roles/c4bf019fc6c44c71a6c984f6d0bab75f => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:16:21.118304 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311b5823-b4bb-4d93-a2ba-fae2f4f22068 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:21.142742 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 40/82] 10.222.0.18 () {60 vars in 1132 bytes} [Wed Nov 2 07:16:20 2022] POST /identity/v3/users => generated 289 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:16:21.144305 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca62d5a-c4ab-49d8-b176-a804ddf7a714 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:21.251216 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311b5823-b4bb-4d93-a2ba-fae2f4f22068 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:21.293417 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33738e6-d12b-4c04-b190-f6caa31781da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:21.294299 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33738e6-d12b-4c04-b190-f6caa31781da None admin] REQUEST_METHOD: `PUT` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:21.294631 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33738e6-d12b-4c04-b190-f6caa31781da None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:21.294909 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33738e6-d12b-4c04-b190-f6caa31781da None admin] PATH_INFO: `/v3/projects/a6b063398180486c85fc258b5a513046/users/7e7f81d50d1a49b3a0076a19ded69866/roles/8cf467a4f06c44e0b77499aeababdcc3` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:21.297611 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33738e6-d12b-4c04-b190-f6caa31781da None admin] RBAC: Authorizing `identity:create_grant(project_id=a6b063398180486c85fc258b5a513046, user_id=7e7f81d50d1a49b3a0076a19ded69866, role_id=8cf467a4f06c44e0b77499aeababdcc3)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:21.299211 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33738e6-d12b-4c04-b190-f6caa31781da None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:21.313681 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca62d5a-c4ab-49d8-b176-a804ddf7a714 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:21.318737 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 43/83] 10.222.0.18 () {58 vars in 1439 bytes} [Wed Nov 2 07:16:21 2022] PUT /identity/v3/projects/a6b063398180486c85fc258b5a513046/users/7e7f81d50d1a49b3a0076a19ded69866/roles/8cf467a4f06c44e0b77499aeababdcc3 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:16:21.319637 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33738e6-d12b-4c04-b190-f6caa31781da None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:21.332747 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:21.361841 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:21.370937 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f42824-d032-4e2c-897c-cf451c09a950 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:21.371434 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f42824-d032-4e2c-897c-cf451c09a950 None admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:21.371619 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f42824-d032-4e2c-897c-cf451c09a950 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:21.371957 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f42824-d032-4e2c-897c-cf451c09a950 None admin] PATH_INFO: `/v3/projects/a6b063398180486c85fc258b5a513046/users/7e7f81d50d1a49b3a0076a19ded69866/roles/b494fedd00f24f0b84a529e8672a156a` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:21.376472 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f42824-d032-4e2c-897c-cf451c09a950 None admin] RBAC: Authorizing `identity:create_grant(project_id=a6b063398180486c85fc258b5a513046, user_id=7e7f81d50d1a49b3a0076a19ded69866, role_id=b494fedd00f24f0b84a529e8672a156a)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:21.377671 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f42824-d032-4e2c-897c-cf451c09a950 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:21.397951 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 41/84] 10.222.0.18 () {58 vars in 1439 bytes} [Wed Nov 2 07:16:21 2022] PUT /identity/v3/projects/a6b063398180486c85fc258b5a513046/users/7e7f81d50d1a49b3a0076a19ded69866/roles/b494fedd00f24f0b84a529e8672a156a => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:16:21.466397 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33738e6-d12b-4c04-b190-f6caa31781da None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:21.510717 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccb2317-e061-4eda-8e84-89e5aa8a98af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:21.511296 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccb2317-e061-4eda-8e84-89e5aa8a98af None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:21.511573 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccb2317-e061-4eda-8e84-89e5aa8a98af None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:21.511851 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccb2317-e061-4eda-8e84-89e5aa8a98af None admin] PATH_INFO: `/v3/projects/2cc14648c48e4b669c12f367ecb8c444` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:21.513125 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ccb2317-e061-4eda-8e84-89e5aa8a98af None admin] RBAC: Authorizing `identity:get_project(project_id=2cc14648c48e4b669c12f367ecb8c444)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:21.514803 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ccb2317-e061-4eda-8e84-89e5aa8a98af None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:21.516701 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 44/85] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Nov 2 07:16:21 2022] GET /identity/v3/projects/2cc14648c48e4b669c12f367ecb8c444 => generated 312 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:21.522963 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f42824-d032-4e2c-897c-cf451c09a950 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:21.663598 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f42824-d032-4e2c-897c-cf451c09a950 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:21.701568 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094e3b9e-c7e2-4656-b419-40273fcb9f0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:21.702313 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094e3b9e-c7e2-4656-b419-40273fcb9f0d None admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:21.702554 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094e3b9e-c7e2-4656-b419-40273fcb9f0d None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:21.702837 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094e3b9e-c7e2-4656-b419-40273fcb9f0d None admin] PATH_INFO: `/v3/projects/2cc14648c48e4b669c12f367ecb8c444/users/7e7f81d50d1a49b3a0076a19ded69866/roles/b494fedd00f24f0b84a529e8672a156a` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:21.704781 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094e3b9e-c7e2-4656-b419-40273fcb9f0d None admin] RBAC: Authorizing `identity:create_grant(project_id=2cc14648c48e4b669c12f367ecb8c444, user_id=7e7f81d50d1a49b3a0076a19ded69866, role_id=b494fedd00f24f0b84a529e8672a156a)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:21.706234 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094e3b9e-c7e2-4656-b419-40273fcb9f0d None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:21.725023 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 42/86] 10.222.0.18 () {58 vars in 1439 bytes} [Wed Nov 2 07:16:21 2022] PUT /identity/v3/projects/2cc14648c48e4b669c12f367ecb8c444/users/7e7f81d50d1a49b3a0076a19ded69866/roles/b494fedd00f24f0b84a529e8672a156a => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:16:23.996517 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6fb42b-2ead-4618-9484-072cfe297f87 None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:23.996997 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6fb42b-2ead-4618-9484-072cfe297f87 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:23.996997 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6fb42b-2ead-4618-9484-072cfe297f87 None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:23.998318 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 45/87] 10.222.0.18 () {54 vars in 927 bytes} [Wed Nov 2 07:16:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:16:24.016071 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083dd514-89ce-4dfc-a4a3-fe99ba22bf5e None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:24.016970 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083dd514-89ce-4dfc-a4a3-fe99ba22bf5e None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:24.017434 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083dd514-89ce-4dfc-a4a3-fe99ba22bf5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:24.023572 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09228167-88ed-4ae3-aa2c-8bf957fd1dd8 None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:24.024580 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09228167-88ed-4ae3-aa2c-8bf957fd1dd8 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:24.025047 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09228167-88ed-4ae3-aa2c-8bf957fd1dd8 None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:24.026588 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 46/88] 10.222.0.18 () {54 vars in 927 bytes} [Wed Nov 2 07:16:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Nov 02 07:16:24.042810 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-083dd514-89ce-4dfc-a4a3-fe99ba22bf5e None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:16:24.044241 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-083dd514-89ce-4dfc-a4a3-fe99ba22bf5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:16:24.050218 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3414e9e5-a57d-4dcd-b9ed-d71798793764 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:24.050740 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3414e9e5-a57d-4dcd-b9ed-d71798793764 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:24.051222 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3414e9e5-a57d-4dcd-b9ed-d71798793764 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:24.082899 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-3414e9e5-a57d-4dcd-b9ed-d71798793764 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:16:24.084599 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-3414e9e5-a57d-4dcd-b9ed-d71798793764 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:16:24.158725 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:24.186094 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 43/89] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Nov 2 07:16:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:16:24.212218 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 47/90] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Nov 2 07:16:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Nov 02 07:16:24.581445 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083dd514-89ce-4dfc-a4a3-fe99ba22bf5e None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:24.712547 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083dd514-89ce-4dfc-a4a3-fe99ba22bf5e None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:24.740282 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3414e9e5-a57d-4dcd-b9ed-d71798793764 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:24.748005 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:24.748448 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d826f5-1c2e-49fd-b8c7-2a0442d26993 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:24.749093 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d826f5-1c2e-49fd-b8c7-2a0442d26993 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:24.749374 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d826f5-1c2e-49fd-b8c7-2a0442d26993 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:24.749649 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d826f5-1c2e-49fd-b8c7-2a0442d26993 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:24.760876 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d826f5-1c2e-49fd-b8c7-2a0442d26993 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:16:24.765882 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-76d826f5-1c2e-49fd-b8c7-2a0442d26993 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Nov 02 07:16:24.765882 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:24.765882 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:16:24.765882 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:16:24.765882 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:16:24.765882 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:16:24.765882 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:16:24.765882 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:16:24.765882 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:16:24.765882 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:16:24.765882 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:16:24.765882 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:16:24.765882 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:16:24.765882 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:16:24.765882 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:16:24.765882 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:16:24.766767 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:16:24.766767 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:16:24.766767 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:16:24.766767 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:16:24.766767 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:16:24.766767 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:16:24.766767 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:16:24.766767 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:16:24.766767 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:16:24.766767 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:16:24.766767 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:16:24.766767 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:16:24.766767 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:16:24.766767 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:16:24.766767 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:16:24.766767 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:16:24.767504 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:16:24.767504 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:16:24.767504 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:16:24.767504 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:16:24.767504 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:16:24.767504 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:16:24.767504 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:16:24.767504 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:16:24.767504 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:16:24.767504 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:16:24.767504 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Nov 02 07:16:24.767504 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:16:24.767504 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 44/91] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Nov 2 07:16:24 2022] GET /identity/v3/roles/reader => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:16:24.772832 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d826f5-1c2e-49fd-b8c7-2a0442d26993 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:24.824372 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:24.883068 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d826f5-1c2e-49fd-b8c7-2a0442d26993 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:24.922011 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1185dfd6-2c57-45b2-a0ba-a2cf8fe17d1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:24.922701 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1185dfd6-2c57-45b2-a0ba-a2cf8fe17d1a None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:24.922986 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1185dfd6-2c57-45b2-a0ba-a2cf8fe17d1a None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:24.923260 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1185dfd6-2c57-45b2-a0ba-a2cf8fe17d1a None admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:24.923937 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1185dfd6-2c57-45b2-a0ba-a2cf8fe17d1a None admin] RBAC: Adding query filter params (name=reader) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:24.924249 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1185dfd6-2c57-45b2-a0ba-a2cf8fe17d1a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:24.925657 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1185dfd6-2c57-45b2-a0ba-a2cf8fe17d1a None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:24.935603 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 45/92] 10.222.0.18 () {56 vars in 1101 bytes} [Wed Nov 2 07:16:24 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:16:24.941177 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7836dc6-dbf3-440e-b826-07b6956c19db None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:24.941471 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7836dc6-dbf3-440e-b826-07b6956c19db None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:24.941749 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7836dc6-dbf3-440e-b826-07b6956c19db None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:24.943559 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 46/93] 10.222.0.18 () {54 vars in 927 bytes} [Wed Nov 2 07:16:24 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:16:24.944578 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7836dc6-dbf3-440e-b826-07b6956c19db None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:25.017126 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3414e9e5-a57d-4dcd-b9ed-d71798793764 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:25.063256 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8723e8-7178-41f8-98d3-5958d774bf57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:25.063932 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8723e8-7178-41f8-98d3-5958d774bf57 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:25.064177 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7836dc6-dbf3-440e-b826-07b6956c19db None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:25.064602 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8723e8-7178-41f8-98d3-5958d774bf57 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:25.064602 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8723e8-7178-41f8-98d3-5958d774bf57 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:25.067559 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:25.076369 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8723e8-7178-41f8-98d3-5958d774bf57 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:16:25.081239 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-cf8723e8-7178-41f8-98d3-5958d774bf57 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 07:16:25.081239 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:25.081239 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:16:25.081239 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:16:25.081239 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:16:25.081239 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:16:25.081239 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:16:25.081239 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:16:25.081239 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:16:25.081239 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:16:25.081239 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:16:25.081239 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:16:25.081239 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:16:25.081239 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:16:25.081239 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:16:25.081239 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:16:25.082077 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:16:25.082077 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:16:25.082077 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:16:25.082077 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:16:25.082077 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:16:25.082077 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:16:25.082077 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:16:25.082077 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:16:25.082077 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:16:25.082077 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:16:25.082077 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:16:25.082077 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Nov 02 07:16:25.082077 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:16:25.082077 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:16:25.082077 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:16:25.082077 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:16:25.083279 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:16:25.083279 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:16:25.083279 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:16:25.083279 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:16:25.083279 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:16:25.083279 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:16:25.083279 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:16:25.083279 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:16:25.083279 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:16:25.083279 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:16:25.083279 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 07:16:25.083279 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Nov 02 07:16:25.083279 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 48/94] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Nov 2 07:16:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 343 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:16:25.083889 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276917f9-c09e-491d-a41e-09d2d2516c74 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:25.084122 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276917f9-c09e-491d-a41e-09d2d2516c74 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:25.084371 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276917f9-c09e-491d-a41e-09d2d2516c74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:25.102794 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-276917f9-c09e-491d-a41e-09d2d2516c74 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:16:25.104208 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-276917f9-c09e-491d-a41e-09d2d2516c74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:16:25.108355 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215f3329-7cc5-4aa1-b486-529eed6c89eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:25.108993 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215f3329-7cc5-4aa1-b486-529eed6c89eb None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:25.109273 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215f3329-7cc5-4aa1-b486-529eed6c89eb None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:25.109546 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215f3329-7cc5-4aa1-b486-529eed6c89eb None admin] PATH_INFO: `/v3/users/cfb3614bc82441bda6fc6213bd2012d9` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:25.123523 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215f3329-7cc5-4aa1-b486-529eed6c89eb None admin] RBAC: Authorizing `identity:get_user(user_id=cfb3614bc82441bda6fc6213bd2012d9)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:25.124961 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215f3329-7cc5-4aa1-b486-529eed6c89eb None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:25.129482 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 47/95] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Nov 2 07:16:24 2022] GET /identity/v3/users/cfb3614bc82441bda6fc6213bd2012d9 => generated 289 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:25.129482 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73f152c-ae84-4cd9-8613-70bbbfa45933 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:25.129482 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73f152c-ae84-4cd9-8613-70bbbfa45933 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:25.129482 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73f152c-ae84-4cd9-8613-70bbbfa45933 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:25.129482 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 48/96] 10.222.0.18 () {54 vars in 927 bytes} [Wed Nov 2 07:16:25 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:16:25.132069 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea44690-3b09-45f2-a9a4-dee06af2b455 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:25.132359 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea44690-3b09-45f2-a9a4-dee06af2b455 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:25.133023 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea44690-3b09-45f2-a9a4-dee06af2b455 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:25.134531 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 49/97] 10.222.0.18 () {54 vars in 927 bytes} [Wed Nov 2 07:16:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:16:25.135187 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea44690-3b09-45f2-a9a4-dee06af2b455 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:25.167759 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:25.176420 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 49/98] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Nov 2 07:16:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:16:25.177939 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328ce491-ba0f-485f-a26c-c09c4750c8a6 None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:25.178348 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328ce491-ba0f-485f-a26c-c09c4750c8a6 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:25.178548 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328ce491-ba0f-485f-a26c-c09c4750c8a6 None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:25.179964 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 50/99] 10.222.0.18 () {54 vars in 927 bytes} [Wed Nov 2 07:16:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:16:25.180649 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328ce491-ba0f-485f-a26c-c09c4750c8a6 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:25.242259 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea44690-3b09-45f2-a9a4-dee06af2b455 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:25.280228 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b746d451-1d8d-494c-ad23-81ef3b6eb1e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:25.280835 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b746d451-1d8d-494c-ad23-81ef3b6eb1e2 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:25.281117 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b746d451-1d8d-494c-ad23-81ef3b6eb1e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:25.281390 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b746d451-1d8d-494c-ad23-81ef3b6eb1e2 None admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:25.282049 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b746d451-1d8d-494c-ad23-81ef3b6eb1e2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:25.282349 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b746d451-1d8d-494c-ad23-81ef3b6eb1e2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:25.283737 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b746d451-1d8d-494c-ad23-81ef3b6eb1e2 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:25.292219 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 50/100] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Nov 2 07:16:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:25.293370 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c375bf2-ce4f-48f1-a347-efc82f8dda9e None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:25.293658 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c375bf2-ce4f-48f1-a347-efc82f8dda9e None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:25.293933 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c375bf2-ce4f-48f1-a347-efc82f8dda9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:25.300173 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328ce491-ba0f-485f-a26c-c09c4750c8a6 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:25.312141 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-6c375bf2-ce4f-48f1-a347-efc82f8dda9e None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:16:25.313338 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-6c375bf2-ce4f-48f1-a347-efc82f8dda9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:16:25.332388 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:25.335548 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d991c16-100c-45ae-b415-199da82b3584 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:25.336745 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d991c16-100c-45ae-b415-199da82b3584 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:25.337062 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d991c16-100c-45ae-b415-199da82b3584 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:25.337340 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d991c16-100c-45ae-b415-199da82b3584 None admin] PATH_INFO: `/v3/projects/a6b063398180486c85fc258b5a513046` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:25.338689 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d991c16-100c-45ae-b415-199da82b3584 None admin] RBAC: Authorizing `identity:get_project(project_id=a6b063398180486c85fc258b5a513046)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:25.340135 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d991c16-100c-45ae-b415-199da82b3584 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:25.346331 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 51/101] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Nov 2 07:16:25 2022] GET /identity/v3/projects/a6b063398180486c85fc258b5a513046 => generated 298 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:25.346331 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d91e9f7-476d-4f7b-84a0-a582368e24a2 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:25.346331 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d91e9f7-476d-4f7b-84a0-a582368e24a2 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:25.346331 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d91e9f7-476d-4f7b-84a0-a582368e24a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:25.363230 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-9d91e9f7-476d-4f7b-84a0-a582368e24a2 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:16:25.364399 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-9d91e9f7-476d-4f7b-84a0-a582368e24a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:16:25.398587 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 51/102] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Nov 2 07:16:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:16:25.398587 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a4e2ca-516c-4fe7-9c96-5a5c67945f0f None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:25.398587 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a4e2ca-516c-4fe7-9c96-5a5c67945f0f None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:25.398587 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a4e2ca-516c-4fe7-9c96-5a5c67945f0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:25.418336 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-e1a4e2ca-516c-4fe7-9c96-5a5c67945f0f None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:16:25.420436 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-e1a4e2ca-516c-4fe7-9c96-5a5c67945f0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:16:25.437555 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:25.445586 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 52/103] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Nov 2 07:16:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:16:25.446798 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d91e9f7-476d-4f7b-84a0-a582368e24a2 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:25.502897 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 52/104] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Nov 2 07:16:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:16:25.507238 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a4e2ca-516c-4fe7-9c96-5a5c67945f0f None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:25.567856 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d91e9f7-476d-4f7b-84a0-a582368e24a2 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:25.606327 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7279fd5d-cba9-4555-a14f-b4ba7c8cc497 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:25.606995 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7279fd5d-cba9-4555-a14f-b4ba7c8cc497 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:25.607272 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7279fd5d-cba9-4555-a14f-b4ba7c8cc497 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:25.607544 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7279fd5d-cba9-4555-a14f-b4ba7c8cc497 None admin] PATH_INFO: `/v3/users/b87b731cb4ea4070ae4dad0c3c393570` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:25.608908 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7279fd5d-cba9-4555-a14f-b4ba7c8cc497 None admin] RBAC: Authorizing `identity:get_user(user_id=b87b731cb4ea4070ae4dad0c3c393570)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:25.610876 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7279fd5d-cba9-4555-a14f-b4ba7c8cc497 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:25.612602 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 53/105] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Nov 2 07:16:25 2022] GET /identity/v3/users/b87b731cb4ea4070ae4dad0c3c393570 => generated 247 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:25.613506 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7279fd5d-cba9-4555-a14f-b4ba7c8cc497 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:25.693927 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:25.696203 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:25.723201 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a4e2ca-516c-4fe7-9c96-5a5c67945f0f None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:25.730208 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7279fd5d-cba9-4555-a14f-b4ba7c8cc497 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:25.764009 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607fb59c-dd98-4661-b6c1-5d856d798548 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:25.764679 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607fb59c-dd98-4661-b6c1-5d856d798548 None admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:25.764958 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607fb59c-dd98-4661-b6c1-5d856d798548 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:25.765229 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607fb59c-dd98-4661-b6c1-5d856d798548 None admin] PATH_INFO: `/v3/projects/a6b063398180486c85fc258b5a513046/users/cfb3614bc82441bda6fc6213bd2012d9/roles/979bdb3f4d8d4d97b1729d3162372a09` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:25.766688 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69439a8c-2295-461a-903a-05dc0a9aec79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:25.767095 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607fb59c-dd98-4661-b6c1-5d856d798548 None admin] RBAC: Authorizing `identity:create_grant(project_id=a6b063398180486c85fc258b5a513046, user_id=cfb3614bc82441bda6fc6213bd2012d9, role_id=979bdb3f4d8d4d97b1729d3162372a09)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:25.767703 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69439a8c-2295-461a-903a-05dc0a9aec79 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:25.768148 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69439a8c-2295-461a-903a-05dc0a9aec79 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:25.768529 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607fb59c-dd98-4661-b6c1-5d856d798548 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:25.768774 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69439a8c-2295-461a-903a-05dc0a9aec79 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:25.770019 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69439a8c-2295-461a-903a-05dc0a9aec79 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:25.771695 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69439a8c-2295-461a-903a-05dc0a9aec79 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:25.773357 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 54/106] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Nov 2 07:16:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:25.774190 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69439a8c-2295-461a-903a-05dc0a9aec79 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:25.782957 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 53/107] 10.222.0.18 () {58 vars in 1439 bytes} [Wed Nov 2 07:16:25 2022] PUT /identity/v3/projects/a6b063398180486c85fc258b5a513046/users/cfb3614bc82441bda6fc6213bd2012d9/roles/979bdb3f4d8d4d97b1729d3162372a09 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:16:25.784021 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607fb59c-dd98-4661-b6c1-5d856d798548 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:25.903946 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69439a8c-2295-461a-903a-05dc0a9aec79 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:25.908574 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607fb59c-dd98-4661-b6c1-5d856d798548 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:25.937828 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:25.947190 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadaf251-ddb1-4417-9395-359df50996e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:25.947738 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3795a4b4-4b35-4052-98f4-9d5cfa0351b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:25.947854 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadaf251-ddb1-4417-9395-359df50996e3 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:25.948037 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadaf251-ddb1-4417-9395-359df50996e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:25.948312 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadaf251-ddb1-4417-9395-359df50996e3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:25.948427 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3795a4b4-4b35-4052-98f4-9d5cfa0351b0 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:25.948643 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3795a4b4-4b35-4052-98f4-9d5cfa0351b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:25.948912 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3795a4b4-4b35-4052-98f4-9d5cfa0351b0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:25.961886 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3795a4b4-4b35-4052-98f4-9d5cfa0351b0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:16:25.967611 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-3795a4b4-4b35-4052-98f4-9d5cfa0351b0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Nov 02 07:16:25.967611 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:25.967611 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:16:25.967611 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:16:25.967611 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:16:25.967611 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:16:25.967611 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:16:25.967611 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:16:25.967611 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:16:25.967611 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:16:25.967611 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:16:25.967611 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:16:25.967611 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:16:25.967611 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:16:25.967611 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:16:25.967611 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:16:25.968589 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:16:25.968589 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:16:25.968589 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:16:25.968589 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:16:25.968589 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:16:25.968589 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:16:25.968589 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:16:25.968589 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:16:25.968589 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:16:25.968589 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:16:25.968589 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:16:25.968589 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:16:25.968589 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:16:25.968589 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:16:25.968589 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:16:25.968589 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:16:25.969593 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:16:25.969593 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:16:25.969593 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:16:25.969593 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:16:25.969593 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:16:25.969593 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:16:25.969593 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:16:25.969593 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:16:25.969593 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:16:25.969593 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:16:25.969593 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Nov 02 07:16:25.969593 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:16:25.969593 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 54/108] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Nov 2 07:16:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:16:25.969593 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3795a4b4-4b35-4052-98f4-9d5cfa0351b0 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:25.980306 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:26.028562 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eadaf251-ddb1-4417-9395-359df50996e3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:26.030050 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eadaf251-ddb1-4417-9395-359df50996e3 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:26.087118 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3795a4b4-4b35-4052-98f4-9d5cfa0351b0 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:26.121779 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf15270-2a18-400b-9c51-7f66cc36abaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:26.122438 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf15270-2a18-400b-9c51-7f66cc36abaf None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:26.122716 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf15270-2a18-400b-9c51-7f66cc36abaf None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:26.122989 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf15270-2a18-400b-9c51-7f66cc36abaf None admin] PATH_INFO: `/v3/roles/member` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:26.133742 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf15270-2a18-400b-9c51-7f66cc36abaf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:16:26.133957 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 55/109] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Nov 2 07:16:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Nov 02 07:16:26.134844 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadaf251-ddb1-4417-9395-359df50996e3 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:26.138999 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-7cf15270-2a18-400b-9c51-7f66cc36abaf None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Nov 02 07:16:26.138999 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:26.138999 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:16:26.138999 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:16:26.138999 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:16:26.138999 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:16:26.138999 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:16:26.138999 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:16:26.138999 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:16:26.138999 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:16:26.138999 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:16:26.138999 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:16:26.138999 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:16:26.138999 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:16:26.138999 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:16:26.138999 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:16:26.139861 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:16:26.139861 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:16:26.139861 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:16:26.139861 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:16:26.139861 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:16:26.139861 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:16:26.139861 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:16:26.139861 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:16:26.139861 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:16:26.139861 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:16:26.139861 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:16:26.139861 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:16:26.139861 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:16:26.139861 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:16:26.139861 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:16:26.139861 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:16:26.140621 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:16:26.140621 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:16:26.140621 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:16:26.140621 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:16:26.140621 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:16:26.140621 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:16:26.140621 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:16:26.140621 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:16:26.140621 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:16:26.140621 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:16:26.140621 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Nov 02 07:16:26.140621 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:16:26.144785 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 55/110] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Nov 2 07:16:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:16:26.145620 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf15270-2a18-400b-9c51-7f66cc36abaf None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:26.167653 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:26.214279 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:26.243096 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadaf251-ddb1-4417-9395-359df50996e3 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:26.256307 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf15270-2a18-400b-9c51-7f66cc36abaf None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:26.278623 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c67e59e-1301-4f9c-86f1-f3c149b91c65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:26.279511 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c67e59e-1301-4f9c-86f1-f3c149b91c65 None admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:26.279962 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c67e59e-1301-4f9c-86f1-f3c149b91c65 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:26.280403 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c67e59e-1301-4f9c-86f1-f3c149b91c65 None admin] PATH_INFO: `/v3/projects` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:26.281242 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c67e59e-1301-4f9c-86f1-f3c149b91c65 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:26.282867 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c67e59e-1301-4f9c-86f1-f3c149b91c65 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:26.293804 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae68edcb-1107-42b6-b24d-03630141a666 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:26.294473 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae68edcb-1107-42b6-b24d-03630141a666 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:26.294758 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae68edcb-1107-42b6-b24d-03630141a666 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:26.295031 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae68edcb-1107-42b6-b24d-03630141a666 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:26.307008 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae68edcb-1107-42b6-b24d-03630141a666 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:16:26.311448 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 56/111] 10.222.0.18 () {60 vars in 1141 bytes} [Wed Nov 2 07:16:26 2022] POST /identity/v3/projects => generated 302 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:16:26.311880 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-ae68edcb-1107-42b6-b24d-03630141a666 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Nov 02 07:16:26.311880 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:26.311880 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:16:26.311880 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:16:26.311880 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:16:26.311880 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:16:26.311880 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:16:26.311880 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:16:26.311880 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:16:26.311880 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:16:26.311880 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:16:26.311880 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:16:26.311880 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:16:26.311880 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:16:26.311880 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:16:26.311880 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:16:26.312701 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:16:26.312701 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:16:26.312701 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:16:26.312701 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:16:26.312701 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:16:26.312701 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:16:26.312701 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:16:26.312701 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:16:26.312701 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:16:26.312701 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:16:26.312701 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:16:26.312701 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:16:26.312701 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:16:26.312701 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:16:26.312701 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:16:26.312701 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:16:26.313474 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:16:26.313474 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:16:26.313474 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:16:26.313474 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:16:26.313474 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:16:26.313474 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:16:26.313474 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:16:26.313474 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:16:26.313474 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:16:26.313474 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:16:26.313474 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Nov 02 07:16:26.313474 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:16:26.313474 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 56/112] 10.222.0.18 () {56 vars in 1114 bytes} [Wed Nov 2 07:16:26 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:16:26.314082 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae68edcb-1107-42b6-b24d-03630141a666 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:26.315243 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c67e59e-1301-4f9c-86f1-f3c149b91c65 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:26.452406 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae68edcb-1107-42b6-b24d-03630141a666 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:26.453022 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:26.480727 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c67e59e-1301-4f9c-86f1-f3c149b91c65 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:26.483208 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:26.498003 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209fc9aa-7862-4104-833b-87106016c5c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:26.499132 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209fc9aa-7862-4104-833b-87106016c5c6 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:26.499596 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209fc9aa-7862-4104-833b-87106016c5c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:26.500037 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209fc9aa-7862-4104-833b-87106016c5c6 None admin] PATH_INFO: `/v3/projects/a6b063398180486c85fc258b5a513046` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:26.501504 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209fc9aa-7862-4104-833b-87106016c5c6 None admin] RBAC: Authorizing `identity:get_project(project_id=a6b063398180486c85fc258b5a513046)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:26.503291 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209fc9aa-7862-4104-833b-87106016c5c6 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:26.505248 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 57/113] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Nov 2 07:16:26 2022] GET /identity/v3/projects/a6b063398180486c85fc258b5a513046 => generated 298 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:26.506242 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209fc9aa-7862-4104-833b-87106016c5c6 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:26.522555 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7a49f7-92c1-481e-ad80-a8a109baa6b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:26.523519 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7a49f7-92c1-481e-ad80-a8a109baa6b1 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:26.523622 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7a49f7-92c1-481e-ad80-a8a109baa6b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:26.524049 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7a49f7-92c1-481e-ad80-a8a109baa6b1 None admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:26.525032 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7a49f7-92c1-481e-ad80-a8a109baa6b1 None admin] RBAC: Adding query filter params (name=member) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:26.525341 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7a49f7-92c1-481e-ad80-a8a109baa6b1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:26.527677 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7a49f7-92c1-481e-ad80-a8a109baa6b1 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:26.536657 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 57/114] 10.222.0.18 () {56 vars in 1101 bytes} [Wed Nov 2 07:16:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:26.537805 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7a49f7-92c1-481e-ad80-a8a109baa6b1 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:26.633070 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209fc9aa-7862-4104-833b-87106016c5c6 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:26.679088 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7a49f7-92c1-481e-ad80-a8a109baa6b1 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:26.686273 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac2120f-9a75-4b60-b831-b5d80dd29299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:26.686935 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac2120f-9a75-4b60-b831-b5d80dd29299 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:26.687216 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac2120f-9a75-4b60-b831-b5d80dd29299 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:26.687488 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac2120f-9a75-4b60-b831-b5d80dd29299 None admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:26.688756 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac2120f-9a75-4b60-b831-b5d80dd29299 None admin] RBAC: Adding query filter params (name=member) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:26.689065 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac2120f-9a75-4b60-b831-b5d80dd29299 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:26.691040 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac2120f-9a75-4b60-b831-b5d80dd29299 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:26.704500 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 58/115] 10.222.0.18 () {56 vars in 1101 bytes} [Wed Nov 2 07:16:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:26.705996 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac2120f-9a75-4b60-b831-b5d80dd29299 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:26.726977 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928d159a-f8db-4a11-b0b4-6c37ede67076 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:26.728395 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928d159a-f8db-4a11-b0b4-6c37ede67076 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:26.729300 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928d159a-f8db-4a11-b0b4-6c37ede67076 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:26.729677 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928d159a-f8db-4a11-b0b4-6c37ede67076 None admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:26.730801 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928d159a-f8db-4a11-b0b4-6c37ede67076 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:26.731173 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928d159a-f8db-4a11-b0b4-6c37ede67076 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:26.733019 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928d159a-f8db-4a11-b0b4-6c37ede67076 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:26.745221 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 58/116] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Nov 2 07:16:26 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:26.746382 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928d159a-f8db-4a11-b0b4-6c37ede67076 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:26.751779 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:26.781064 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:26.845707 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac2120f-9a75-4b60-b831-b5d80dd29299 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:26.862783 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928d159a-f8db-4a11-b0b4-6c37ede67076 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:26.894378 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa0110c-ddaf-4feb-945a-5e659b885c76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:26.895431 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa0110c-ddaf-4feb-945a-5e659b885c76 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:26.895916 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa0110c-ddaf-4feb-945a-5e659b885c76 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:26.896427 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa0110c-ddaf-4feb-945a-5e659b885c76 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:26.897552 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa0110c-ddaf-4feb-945a-5e659b885c76 None admin] RBAC: Adding query filter params (role.id=c4bf019fc6c44c71a6c984f6d0bab75f, scope.project.id=a6b063398180486c85fc258b5a513046, user.id=b87b731cb4ea4070ae4dad0c3c393570) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:26.898019 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa0110c-ddaf-4feb-945a-5e659b885c76 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:26.898923 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61efabaf-2ceb-45f6-936a-15c52848fa7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:26.899682 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61efabaf-2ceb-45f6-936a-15c52848fa7b None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:26.900062 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61efabaf-2ceb-45f6-936a-15c52848fa7b None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:26.900436 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61efabaf-2ceb-45f6-936a-15c52848fa7b None admin] PATH_INFO: `/v3/users/7e7f81d50d1a49b3a0076a19ded69866` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:26.900702 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa0110c-ddaf-4feb-945a-5e659b885c76 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:26.901765 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61efabaf-2ceb-45f6-936a-15c52848fa7b None admin] RBAC: Authorizing `identity:get_user(user_id=7e7f81d50d1a49b3a0076a19ded69866)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:26.903335 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61efabaf-2ceb-45f6-936a-15c52848fa7b None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:26.905027 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 59/117] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Nov 2 07:16:26 2022] GET /identity/v3/users/7e7f81d50d1a49b3a0076a19ded69866 => generated 275 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:26.905881 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61efabaf-2ceb-45f6-936a-15c52848fa7b None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:26.908094 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 59/118] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Nov 2 07:16:26 2022] GET /identity/v3/role_assignments?user.id=b87b731cb4ea4070ae4dad0c3c393570&scope.project.id=a6b063398180486c85fc258b5a513046&role.id=c4bf019fc6c44c71a6c984f6d0bab75f => generated 608 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:26.910442 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa0110c-ddaf-4feb-945a-5e659b885c76 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:26.986383 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:27.037597 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61efabaf-2ceb-45f6-936a-15c52848fa7b None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:27.059004 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa0110c-ddaf-4feb-945a-5e659b885c76 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:27.080766 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:27.110935 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890d6859-9387-4035-9dca-911018bab0ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:27.111827 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890d6859-9387-4035-9dca-911018bab0ca None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:27.112251 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890d6859-9387-4035-9dca-911018bab0ca None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:27.112665 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890d6859-9387-4035-9dca-911018bab0ca None admin] PATH_INFO: `/v3/users/7e7f81d50d1a49b3a0076a19ded69866` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:27.114021 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890d6859-9387-4035-9dca-911018bab0ca None admin] RBAC: Authorizing `identity:get_user(user_id=7e7f81d50d1a49b3a0076a19ded69866)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:27.115784 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c5e4af-b480-4177-b9d5-66b9a633f6b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:27.116348 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890d6859-9387-4035-9dca-911018bab0ca None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:27.116454 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c5e4af-b480-4177-b9d5-66b9a633f6b5 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:27.116723 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c5e4af-b480-4177-b9d5-66b9a633f6b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:27.116997 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c5e4af-b480-4177-b9d5-66b9a633f6b5 None admin] PATH_INFO: `/v3/users/7e7f81d50d1a49b3a0076a19ded69866` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:27.119367 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 60/119] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Nov 2 07:16:26 2022] GET /identity/v3/users/7e7f81d50d1a49b3a0076a19ded69866 => generated 275 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:27.120387 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890d6859-9387-4035-9dca-911018bab0ca None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:27.120632 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c5e4af-b480-4177-b9d5-66b9a633f6b5 None admin] RBAC: Authorizing `identity:get_user(user_id=7e7f81d50d1a49b3a0076a19ded69866)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:27.124570 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c5e4af-b480-4177-b9d5-66b9a633f6b5 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:27.124570 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 60/120] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Nov 2 07:16:26 2022] GET /identity/v3/users/7e7f81d50d1a49b3a0076a19ded69866 => generated 275 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:27.134173 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c5e4af-b480-4177-b9d5-66b9a633f6b5 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:27.267704 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c5e4af-b480-4177-b9d5-66b9a633f6b5 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:27.269298 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890d6859-9387-4035-9dca-911018bab0ca None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:27.307935 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d84714f-3a81-404e-be6d-12d2304afee5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:27.308590 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d84714f-3a81-404e-be6d-12d2304afee5 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:27.308877 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d84714f-3a81-404e-be6d-12d2304afee5 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:27.309152 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d84714f-3a81-404e-be6d-12d2304afee5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:27.315914 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786ca7e0-a8b8-4c47-8d20-6496b4f8a54e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:27.316527 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786ca7e0-a8b8-4c47-8d20-6496b4f8a54e None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:27.316808 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786ca7e0-a8b8-4c47-8d20-6496b4f8a54e None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:27.317077 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786ca7e0-a8b8-4c47-8d20-6496b4f8a54e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:27.358756 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:27.380502 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:27.399495 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d84714f-3a81-404e-be6d-12d2304afee5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:27.400962 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d84714f-3a81-404e-be6d-12d2304afee5 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:27.406863 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786ca7e0-a8b8-4c47-8d20-6496b4f8a54e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:27.408544 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786ca7e0-a8b8-4c47-8d20-6496b4f8a54e None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:27.514721 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 61/121] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Nov 2 07:16:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Nov 02 07:16:27.515618 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d84714f-3a81-404e-be6d-12d2304afee5 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:27.516528 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 61/122] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Nov 2 07:16:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Nov 02 07:16:27.524541 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786ca7e0-a8b8-4c47-8d20-6496b4f8a54e None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:27.579638 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:27.609496 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:27.630596 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d84714f-3a81-404e-be6d-12d2304afee5 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:27.659314 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786ca7e0-a8b8-4c47-8d20-6496b4f8a54e None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:27.668165 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d562af07-3273-4722-a877-10103668c9c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:27.668797 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d562af07-3273-4722-a877-10103668c9c0 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:27.669083 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d562af07-3273-4722-a877-10103668c9c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:27.669479 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d562af07-3273-4722-a877-10103668c9c0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:27.694237 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcf685e-c859-485b-8c6d-50d94283ca2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:27.695320 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcf685e-c859-485b-8c6d-50d94283ca2f None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:27.695826 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcf685e-c859-485b-8c6d-50d94283ca2f None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:27.696266 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcf685e-c859-485b-8c6d-50d94283ca2f None admin] PATH_INFO: `/v3/projects/a6b063398180486c85fc258b5a513046` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:27.697558 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbcf685e-c859-485b-8c6d-50d94283ca2f None admin] RBAC: Authorizing `identity:get_project(project_id=a6b063398180486c85fc258b5a513046)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:27.699208 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbcf685e-c859-485b-8c6d-50d94283ca2f None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:27.701006 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 62/123] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Nov 2 07:16:27 2022] GET /identity/v3/projects/a6b063398180486c85fc258b5a513046 => generated 298 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:27.706882 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcf685e-c859-485b-8c6d-50d94283ca2f None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:27.740818 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d562af07-3273-4722-a877-10103668c9c0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:27.742878 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d562af07-3273-4722-a877-10103668c9c0 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:27.813802 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcf685e-c859-485b-8c6d-50d94283ca2f None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:27.824319 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:27.828503 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:27.849660 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 62/124] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Nov 2 07:16:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Nov 02 07:16:27.849660 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d562af07-3273-4722-a877-10103668c9c0 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:27.862258 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38ec010-d8a3-4ef7-b280-5f6fcb464da7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:27.863184 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38ec010-d8a3-4ef7-b280-5f6fcb464da7 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:27.863637 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38ec010-d8a3-4ef7-b280-5f6fcb464da7 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:27.864124 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38ec010-d8a3-4ef7-b280-5f6fcb464da7 None admin] PATH_INFO: `/v3/projects/2cc14648c48e4b669c12f367ecb8c444` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:27.865508 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38ec010-d8a3-4ef7-b280-5f6fcb464da7 None admin] RBAC: Authorizing `identity:get_project(project_id=2cc14648c48e4b669c12f367ecb8c444)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:27.867722 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38ec010-d8a3-4ef7-b280-5f6fcb464da7 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:27.869693 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 63/125] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Nov 2 07:16:27 2022] GET /identity/v3/projects/2cc14648c48e4b669c12f367ecb8c444 => generated 312 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:27.870768 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38ec010-d8a3-4ef7-b280-5f6fcb464da7 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:27.961521 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d562af07-3273-4722-a877-10103668c9c0 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:27.986492 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38ec010-d8a3-4ef7-b280-5f6fcb464da7 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:27.996533 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4379b7-da0b-4010-b5dd-c8e723d83893 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:27.997138 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4379b7-da0b-4010-b5dd-c8e723d83893 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:27.997423 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4379b7-da0b-4010-b5dd-c8e723d83893 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:27.997700 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4379b7-da0b-4010-b5dd-c8e723d83893 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:27.998449 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4379b7-da0b-4010-b5dd-c8e723d83893 None admin] RBAC: Adding query filter params (role.id=8cf467a4f06c44e0b77499aeababdcc3, scope.project.id=a6b063398180486c85fc258b5a513046, user.id=7e7f81d50d1a49b3a0076a19ded69866) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:27.998758 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4379b7-da0b-4010-b5dd-c8e723d83893 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:28.000188 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4379b7-da0b-4010-b5dd-c8e723d83893 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:28.011239 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 63/126] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Nov 2 07:16:27 2022] GET /identity/v3/role_assignments?user.id=7e7f81d50d1a49b3a0076a19ded69866&scope.project.id=a6b063398180486c85fc258b5a513046&role.id=8cf467a4f06c44e0b77499aeababdcc3 => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:28.011239 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4379b7-da0b-4010-b5dd-c8e723d83893 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:28.057723 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d577dc8-4a4d-4114-aefb-645b80b69434 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:28.058401 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d577dc8-4a4d-4114-aefb-645b80b69434 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:28.058744 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d577dc8-4a4d-4114-aefb-645b80b69434 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:28.059025 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d577dc8-4a4d-4114-aefb-645b80b69434 None admin] PATH_INFO: `/v3/projects/a6b063398180486c85fc258b5a513046` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:28.061600 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d577dc8-4a4d-4114-aefb-645b80b69434 None admin] RBAC: Authorizing `identity:get_project(project_id=a6b063398180486c85fc258b5a513046)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:28.063238 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d577dc8-4a4d-4114-aefb-645b80b69434 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:28.065725 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 64/127] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Nov 2 07:16:27 2022] GET /identity/v3/projects/a6b063398180486c85fc258b5a513046 => generated 298 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:28.075514 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d577dc8-4a4d-4114-aefb-645b80b69434 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:28.121959 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:28.175450 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:28.213836 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4379b7-da0b-4010-b5dd-c8e723d83893 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:28.258559 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4a5ffa-8980-434e-be62-2848d75105fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:28.258998 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d577dc8-4a4d-4114-aefb-645b80b69434 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:28.259209 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4a5ffa-8980-434e-be62-2848d75105fd None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:28.259432 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4a5ffa-8980-434e-be62-2848d75105fd None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:28.259766 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4a5ffa-8980-434e-be62-2848d75105fd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:28.260614 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4a5ffa-8980-434e-be62-2848d75105fd None admin] RBAC: Adding query filter params (role.id=b494fedd00f24f0b84a529e8672a156a, scope.project.id=2cc14648c48e4b669c12f367ecb8c444, user.id=7e7f81d50d1a49b3a0076a19ded69866) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:28.260922 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4a5ffa-8980-434e-be62-2848d75105fd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:28.262852 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4a5ffa-8980-434e-be62-2848d75105fd None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:28.271470 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 64/128] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Nov 2 07:16:28 2022] GET /identity/v3/role_assignments?user.id=7e7f81d50d1a49b3a0076a19ded69866&scope.project.id=2cc14648c48e4b669c12f367ecb8c444&role.id=b494fedd00f24f0b84a529e8672a156a => generated 608 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:28.271470 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca347a90-6784-4e43-aadb-3905dad0412d None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:28.271833 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca347a90-6784-4e43-aadb-3905dad0412d None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:28.272027 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca347a90-6784-4e43-aadb-3905dad0412d None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:28.273508 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 65/129] 10.222.0.18 () {54 vars in 927 bytes} [Wed Nov 2 07:16:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:16:28.281834 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d86ad0-78eb-4a9a-84b0-523b7a53e3c0 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:28.282150 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d86ad0-78eb-4a9a-84b0-523b7a53e3c0 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:28.282444 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d86ad0-78eb-4a9a-84b0-523b7a53e3c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:28.302924 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-a9d86ad0-78eb-4a9a-84b0-523b7a53e3c0 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:16:28.305586 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-a9d86ad0-78eb-4a9a-84b0-523b7a53e3c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:16:28.315008 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aad661b-c821-4cdf-a02c-5c34dbf0a331 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:28.316056 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aad661b-c821-4cdf-a02c-5c34dbf0a331 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:28.316627 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aad661b-c821-4cdf-a02c-5c34dbf0a331 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:28.317098 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aad661b-c821-4cdf-a02c-5c34dbf0a331 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:28.318064 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aad661b-c821-4cdf-a02c-5c34dbf0a331 None admin] RBAC: Adding query filter params (role.id=b494fedd00f24f0b84a529e8672a156a, scope.project.id=a6b063398180486c85fc258b5a513046, user.id=7e7f81d50d1a49b3a0076a19ded69866) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:28.318619 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aad661b-c821-4cdf-a02c-5c34dbf0a331 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:28.321173 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aad661b-c821-4cdf-a02c-5c34dbf0a331 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:28.330963 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 65/130] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Nov 2 07:16:28 2022] GET /identity/v3/role_assignments?user.id=7e7f81d50d1a49b3a0076a19ded69866&scope.project.id=a6b063398180486c85fc258b5a513046&role.id=b494fedd00f24f0b84a529e8672a156a => generated 608 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:28.395824 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 66/131] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Nov 2 07:16:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:16:28.686512 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aad661b-c821-4cdf-a02c-5c34dbf0a331 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:28.717716 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa241a6-1e42-4a8c-94ac-fae8be8edbf3 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:28.718004 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa241a6-1e42-4a8c-94ac-fae8be8edbf3 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:28.718276 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa241a6-1e42-4a8c-94ac-fae8be8edbf3 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:28.719780 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 67/132] 10.222.0.18 () {54 vars in 927 bytes} [Wed Nov 2 07:16:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Nov 02 07:16:28.724632 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bc2644-53d1-4b60-8230-1f041cfb190b None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:28.724920 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bc2644-53d1-4b60-8230-1f041cfb190b None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:28.725198 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bc2644-53d1-4b60-8230-1f041cfb190b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:28.743890 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-79bc2644-53d1-4b60-8230-1f041cfb190b None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:16:28.745071 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-79bc2644-53d1-4b60-8230-1f041cfb190b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:16:28.784196 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:28.789523 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aad661b-c821-4cdf-a02c-5c34dbf0a331 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:28.798221 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:28.810670 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 68/133] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Nov 2 07:16:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Nov 02 07:16:28.825362 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267bab90-805c-4d21-b390-67eaa55af3f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:28.825998 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267bab90-805c-4d21-b390-67eaa55af3f2 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:28.826282 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267bab90-805c-4d21-b390-67eaa55af3f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:28.826587 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267bab90-805c-4d21-b390-67eaa55af3f2 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:28.837104 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267bab90-805c-4d21-b390-67eaa55af3f2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:16:28.842347 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-267bab90-805c-4d21-b390-67eaa55af3f2 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Nov 02 07:16:28.842347 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:28.842347 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:16:28.842347 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:16:28.842347 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:16:28.842347 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:16:28.842347 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:16:28.842347 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:16:28.842347 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:16:28.842347 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:16:28.842347 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:16:28.842347 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:16:28.842347 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:16:28.842347 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:16:28.842347 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:16:28.842347 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:16:28.843235 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:16:28.843235 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:16:28.843235 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:16:28.843235 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:16:28.843235 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:16:28.843235 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:16:28.843235 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:16:28.843235 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:16:28.843235 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:16:28.843235 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:16:28.843235 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:16:28.843235 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Nov 02 07:16:28.843235 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:16:28.843235 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:16:28.843235 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:16:28.843235 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:16:28.843991 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:16:28.843991 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:16:28.843991 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:16:28.843991 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:16:28.843991 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:16:28.843991 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:16:28.843991 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:16:28.843991 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:16:28.843991 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:16:28.843991 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:16:28.843991 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Nov 02 07:16:28.843991 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Nov 02 07:16:28.843991 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 66/134] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Nov 2 07:16:28 2022] GET /identity/v3/roles/reader => generated 84 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:16:28.849216 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bc2644-53d1-4b60-8230-1f041cfb190b None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:28.946731 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bc2644-53d1-4b60-8230-1f041cfb190b None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:28.980380 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e431f660-3c28-4eec-aaa3-b0c194fd82f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:28.980981 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e431f660-3c28-4eec-aaa3-b0c194fd82f3 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:28.981261 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e431f660-3c28-4eec-aaa3-b0c194fd82f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:28.981534 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e431f660-3c28-4eec-aaa3-b0c194fd82f3 None admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:28.982186 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e431f660-3c28-4eec-aaa3-b0c194fd82f3 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:28.982521 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e431f660-3c28-4eec-aaa3-b0c194fd82f3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:28.983892 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e431f660-3c28-4eec-aaa3-b0c194fd82f3 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:28.991758 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 69/135] 10.222.0.18 () {56 vars in 1101 bytes} [Wed Nov 2 07:16:28 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:16:28.997782 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267bab90-805c-4d21-b390-67eaa55af3f2 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:29.082597 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e431f660-3c28-4eec-aaa3-b0c194fd82f3 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:29.094560 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267bab90-805c-4d21-b390-67eaa55af3f2 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:29.117038 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:29.128049 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db480a7-cc3f-4c09-a760-d29c3511a4f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:29.128683 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db480a7-cc3f-4c09-a760-d29c3511a4f2 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:29.128967 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db480a7-cc3f-4c09-a760-d29c3511a4f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:29.129243 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db480a7-cc3f-4c09-a760-d29c3511a4f2 None admin] PATH_INFO: `/v3/users/cfb3614bc82441bda6fc6213bd2012d9` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:29.130350 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db480a7-cc3f-4c09-a760-d29c3511a4f2 None admin] RBAC: Authorizing `identity:get_user(user_id=cfb3614bc82441bda6fc6213bd2012d9)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:29.131835 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db480a7-cc3f-4c09-a760-d29c3511a4f2 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:29.133973 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 67/136] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Nov 2 07:16:28 2022] GET /identity/v3/users/cfb3614bc82441bda6fc6213bd2012d9 => generated 289 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:29.139374 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db480a7-cc3f-4c09-a760-d29c3511a4f2 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:29.175146 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:29.182268 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e431f660-3c28-4eec-aaa3-b0c194fd82f3 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:29.215475 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492ca03f-e615-48cb-9a68-06c09bab1299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:29.216040 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492ca03f-e615-48cb-9a68-06c09bab1299 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:29.216319 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492ca03f-e615-48cb-9a68-06c09bab1299 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:29.216591 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492ca03f-e615-48cb-9a68-06c09bab1299 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:29.217684 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492ca03f-e615-48cb-9a68-06c09bab1299 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:29.219152 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492ca03f-e615-48cb-9a68-06c09bab1299 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:29.220688 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 70/137] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Nov 2 07:16:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:16:29.226154 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492ca03f-e615-48cb-9a68-06c09bab1299 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:29.240878 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db480a7-cc3f-4c09-a760-d29c3511a4f2 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:29.275773 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af47f1ca-8e72-4399-91c7-a7026c21a003 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:29.276378 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af47f1ca-8e72-4399-91c7-a7026c21a003 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:29.276659 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af47f1ca-8e72-4399-91c7-a7026c21a003 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:29.276941 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af47f1ca-8e72-4399-91c7-a7026c21a003 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:29.325609 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492ca03f-e615-48cb-9a68-06c09bab1299 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:29.328317 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:29.345697 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af47f1ca-8e72-4399-91c7-a7026c21a003 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:29.347199 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af47f1ca-8e72-4399-91c7-a7026c21a003 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:29.359818 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0580673e-a55d-42a2-967d-0d15edefd486 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:29.360427 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0580673e-a55d-42a2-967d-0d15edefd486 None admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:29.360707 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0580673e-a55d-42a2-967d-0d15edefd486 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:29.360979 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0580673e-a55d-42a2-967d-0d15edefd486 None admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:29.361619 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0580673e-a55d-42a2-967d-0d15edefd486 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:29.363066 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0580673e-a55d-42a2-967d-0d15edefd486 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:29.382805 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:29.384130 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 71/138] 10.222.0.18 () {60 vars in 1132 bytes} [Wed Nov 2 07:16:29 2022] POST /identity/v3/users => generated 283 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 07:16:29.455484 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 68/139] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Nov 2 07:16:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Nov 02 07:16:29.461885 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0580673e-a55d-42a2-967d-0d15edefd486 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:29.567103 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0580673e-a55d-42a2-967d-0d15edefd486 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:29.600636 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa8744a-8bbb-4502-8a65-9805c8178a64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:29.601286 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa8744a-8bbb-4502-8a65-9805c8178a64 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:29.601575 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa8744a-8bbb-4502-8a65-9805c8178a64 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:29.601846 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa8744a-8bbb-4502-8a65-9805c8178a64 None admin] PATH_INFO: `/v3/projects/a6b063398180486c85fc258b5a513046` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:29.602982 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa8744a-8bbb-4502-8a65-9805c8178a64 None admin] RBAC: Authorizing `identity:get_project(project_id=a6b063398180486c85fc258b5a513046)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:29.604671 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa8744a-8bbb-4502-8a65-9805c8178a64 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:29.606284 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 72/140] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Nov 2 07:16:29 2022] GET /identity/v3/projects/a6b063398180486c85fc258b5a513046 => generated 298 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:29.612233 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af47f1ca-8e72-4399-91c7-a7026c21a003 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:29.717966 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af47f1ca-8e72-4399-91c7-a7026c21a003 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:29.752259 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a509187-5a17-4679-8a35-f547dd15303e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:29.752896 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a509187-5a17-4679-8a35-f547dd15303e None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:29.753180 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a509187-5a17-4679-8a35-f547dd15303e None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:29.753452 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a509187-5a17-4679-8a35-f547dd15303e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:29.754175 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a509187-5a17-4679-8a35-f547dd15303e None admin] RBAC: Adding query filter params (role.id=979bdb3f4d8d4d97b1729d3162372a09, scope.project.id=a6b063398180486c85fc258b5a513046, user.id=cfb3614bc82441bda6fc6213bd2012d9) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:29.754505 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a509187-5a17-4679-8a35-f547dd15303e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:29.756127 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a509187-5a17-4679-8a35-f547dd15303e None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:29.760068 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:29.763881 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 69/141] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Nov 2 07:16:29 2022] GET /identity/v3/role_assignments?user.id=cfb3614bc82441bda6fc6213bd2012d9&scope.project.id=a6b063398180486c85fc258b5a513046&role.id=979bdb3f4d8d4d97b1729d3162372a09 => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:31.635176 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e36ab1-81f0-4cca-bfa0-a942a8253621 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:31.635741 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e36ab1-81f0-4cca-bfa0-a942a8253621 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:31.635741 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e36ab1-81f0-4cca-bfa0-a942a8253621 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:31.637098 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 73/142] 10.222.0.18 () {54 vars in 927 bytes} [Wed Nov 2 07:16:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:16:31.642655 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c1fdf9-4269-4df8-872b-dfca86bd96fc None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:31.642655 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c1fdf9-4269-4df8-872b-dfca86bd96fc None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:31.643158 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c1fdf9-4269-4df8-872b-dfca86bd96fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:31.663364 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-47c1fdf9-4269-4df8-872b-dfca86bd96fc None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:16:31.664949 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-47c1fdf9-4269-4df8-872b-dfca86bd96fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:16:31.742430 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 70/143] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Nov 2 07:16:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:16:31.779520 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e990ad-cc35-42a3-a63b-a6e3b0164971 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:31.779806 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e990ad-cc35-42a3-a63b-a6e3b0164971 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:31.780267 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e990ad-cc35-42a3-a63b-a6e3b0164971 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:31.781604 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 74/144] 10.222.0.18 () {54 vars in 927 bytes} [Wed Nov 2 07:16:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:16:31.786849 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49989d36-a508-4c5b-8a88-2823d77dafd9 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:31.787143 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49989d36-a508-4c5b-8a88-2823d77dafd9 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:31.787423 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49989d36-a508-4c5b-8a88-2823d77dafd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:31.806468 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-49989d36-a508-4c5b-8a88-2823d77dafd9 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:16:31.807626 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-49989d36-a508-4c5b-8a88-2823d77dafd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:16:31.839319 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f3d079-ba34-4373-8d76-28a504bc6f59 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:31.839615 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f3d079-ba34-4373-8d76-28a504bc6f59 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:31.839881 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f3d079-ba34-4373-8d76-28a504bc6f59 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:31.841393 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 75/145] 10.222.0.18 () {54 vars in 927 bytes} [Wed Nov 2 07:16:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Nov 02 07:16:31.846491 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a35eb16-b0ed-485f-8ac1-764ebb23910d None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:31.846776 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a35eb16-b0ed-485f-8ac1-764ebb23910d None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:31.847041 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a35eb16-b0ed-485f-8ac1-764ebb23910d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:31.866240 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-9a35eb16-b0ed-485f-8ac1-764ebb23910d None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:16:31.868019 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-9a35eb16-b0ed-485f-8ac1-764ebb23910d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:16:31.883992 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 71/146] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Nov 2 07:16:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:16:31.892416 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:31.938814 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 76/147] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Nov 2 07:16:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Nov 02 07:16:32.006995 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49989d36-a508-4c5b-8a88-2823d77dafd9 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:32.068533 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:32.108756 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49989d36-a508-4c5b-8a88-2823d77dafd9 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:32.141436 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101f6246-5472-45e3-8622-99c16b38eb4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:32.142055 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101f6246-5472-45e3-8622-99c16b38eb4a None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:32.142329 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101f6246-5472-45e3-8622-99c16b38eb4a None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:32.142680 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101f6246-5472-45e3-8622-99c16b38eb4a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:32.152934 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101f6246-5472-45e3-8622-99c16b38eb4a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:16:32.157708 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-101f6246-5472-45e3-8622-99c16b38eb4a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 07:16:32.157708 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:32.157708 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:16:32.157708 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:16:32.157708 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:16:32.157708 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:16:32.157708 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:16:32.157708 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:16:32.157708 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:16:32.157708 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:16:32.157708 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:16:32.157708 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:16:32.157708 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:16:32.157708 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:16:32.157708 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:16:32.157708 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:16:32.158603 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:16:32.158603 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:16:32.158603 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:16:32.158603 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:16:32.158603 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:16:32.158603 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:16:32.158603 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:16:32.158603 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:16:32.158603 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:16:32.158603 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:16:32.158603 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:16:32.158603 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Nov 02 07:16:32.158603 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:16:32.158603 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:16:32.158603 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:16:32.158603 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:16:32.159400 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:16:32.159400 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:16:32.159400 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:16:32.159400 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:16:32.159400 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:16:32.159400 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:16:32.159400 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:16:32.159400 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:16:32.159400 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:16:32.159400 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:16:32.159400 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 07:16:32.159400 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Nov 02 07:16:32.160206 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a35eb16-b0ed-485f-8ac1-764ebb23910d None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:32.160282 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 72/148] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Nov 2 07:16:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:16:32.166089 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101f6246-5472-45e3-8622-99c16b38eb4a None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:32.257306 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a35eb16-b0ed-485f-8ac1-764ebb23910d None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:32.266773 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101f6246-5472-45e3-8622-99c16b38eb4a None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:32.287667 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:32.291436 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b14ad4b-94e4-4a71-b1fa-220a7b7cad92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:32.292073 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b14ad4b-94e4-4a71-b1fa-220a7b7cad92 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:32.292349 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b14ad4b-94e4-4a71-b1fa-220a7b7cad92 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:32.292626 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b14ad4b-94e4-4a71-b1fa-220a7b7cad92 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:32.302435 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b14ad4b-94e4-4a71-b1fa-220a7b7cad92 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:16:32.302707 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff779e0-fb5d-4493-bf88-cdc22cdfafd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:32.303307 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff779e0-fb5d-4493-bf88-cdc22cdfafd8 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:32.303585 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff779e0-fb5d-4493-bf88-cdc22cdfafd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:32.303851 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff779e0-fb5d-4493-bf88-cdc22cdfafd8 None admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:32.304543 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff779e0-fb5d-4493-bf88-cdc22cdfafd8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:32.304841 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff779e0-fb5d-4493-bf88-cdc22cdfafd8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:32.306214 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff779e0-fb5d-4493-bf88-cdc22cdfafd8 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:32.307432 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-3b14ad4b-94e4-4a71-b1fa-220a7b7cad92 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Nov 02 07:16:32.307432 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:32.307432 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:16:32.307432 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:16:32.307432 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:16:32.307432 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:16:32.307432 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:16:32.307432 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:16:32.307432 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:16:32.307432 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:16:32.307432 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:16:32.307432 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:16:32.307432 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:16:32.307432 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:16:32.307432 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:16:32.307432 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:16:32.308351 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:16:32.308351 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:16:32.308351 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:16:32.308351 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:16:32.308351 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:16:32.308351 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:16:32.308351 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:16:32.308351 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:16:32.308351 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:16:32.308351 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:16:32.308351 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:16:32.308351 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:16:32.308351 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:16:32.308351 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:16:32.308351 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:16:32.308351 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:16:32.309115 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:16:32.309115 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:16:32.309115 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:16:32.309115 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:16:32.309115 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:16:32.309115 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:16:32.309115 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:16:32.309115 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:16:32.309115 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:16:32.309115 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:16:32.309115 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Nov 02 07:16:32.309115 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:16:32.309115 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 77/149] 10.222.0.18 () {56 vars in 1114 bytes} [Wed Nov 2 07:16:32 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 07:16:32.309825 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b14ad4b-94e4-4a71-b1fa-220a7b7cad92 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:32.321266 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 73/150] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Nov 2 07:16:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:16:32.321266 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff779e0-fb5d-4493-bf88-cdc22cdfafd8 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:32.328949 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:32.410711 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b14ad4b-94e4-4a71-b1fa-220a7b7cad92 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:32.416701 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff779e0-fb5d-4493-bf88-cdc22cdfafd8 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:32.444528 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5dc08f-bb7c-4e64-925c-7123b8e9e5b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:32.445219 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5dc08f-bb7c-4e64-925c-7123b8e9e5b3 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:32.445496 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5dc08f-bb7c-4e64-925c-7123b8e9e5b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:32.445768 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5dc08f-bb7c-4e64-925c-7123b8e9e5b3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:32.446895 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5dc08f-bb7c-4e64-925c-7123b8e9e5b3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:32.448343 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5dc08f-bb7c-4e64-925c-7123b8e9e5b3 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:32.449588 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ef5e87-d19f-466a-bdf8-4bb648eddb8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:32.449884 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 78/151] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Nov 2 07:16:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:32.450250 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ef5e87-d19f-466a-bdf8-4bb648eddb8d None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:32.450572 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ef5e87-d19f-466a-bdf8-4bb648eddb8d None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:32.450736 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5dc08f-bb7c-4e64-925c-7123b8e9e5b3 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:32.450839 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ef5e87-d19f-466a-bdf8-4bb648eddb8d None admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:32.451472 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ef5e87-d19f-466a-bdf8-4bb648eddb8d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:32.451772 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ef5e87-d19f-466a-bdf8-4bb648eddb8d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:32.453119 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ef5e87-d19f-466a-bdf8-4bb648eddb8d None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:32.461508 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 74/152] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Nov 2 07:16:32 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:32.462275 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ef5e87-d19f-466a-bdf8-4bb648eddb8d None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:32.507080 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:32.536146 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:32.551115 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5dc08f-bb7c-4e64-925c-7123b8e9e5b3 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:32.562794 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ef5e87-d19f-466a-bdf8-4bb648eddb8d None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:32.584625 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c80c086-1ae5-4245-b46c-a6d61c2437f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:32.585250 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c80c086-1ae5-4245-b46c-a6d61c2437f0 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:32.585528 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c80c086-1ae5-4245-b46c-a6d61c2437f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:32.585799 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c80c086-1ae5-4245-b46c-a6d61c2437f0 None admin] PATH_INFO: `/v3/users/93596e79352c4e77bb8411f959b7e723` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:32.596018 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d93888a-9672-4e0d-814f-292994fb82de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:32.596621 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d93888a-9672-4e0d-814f-292994fb82de None admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:32.596904 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d93888a-9672-4e0d-814f-292994fb82de None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:32.597177 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d93888a-9672-4e0d-814f-292994fb82de None admin] PATH_INFO: `/v3/users` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:32.597817 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d93888a-9672-4e0d-814f-292994fb82de None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:32.598440 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c80c086-1ae5-4245-b46c-a6d61c2437f0 None admin] RBAC: Authorizing `identity:get_user(user_id=93596e79352c4e77bb8411f959b7e723)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:32.599292 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d93888a-9672-4e0d-814f-292994fb82de None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:32.599858 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c80c086-1ae5-4245-b46c-a6d61c2437f0 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:32.601964 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 79/153] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Nov 2 07:16:32 2022] GET /identity/v3/users/93596e79352c4e77bb8411f959b7e723 => generated 283 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:32.604091 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c80c086-1ae5-4245-b46c-a6d61c2437f0 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:32.621180 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 75/154] 10.222.0.18 () {60 vars in 1132 bytes} [Wed Nov 2 07:16:32 2022] POST /identity/v3/users => generated 297 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:16:32.622038 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d93888a-9672-4e0d-814f-292994fb82de None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:32.735822 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c80c086-1ae5-4245-b46c-a6d61c2437f0 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:32.747994 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:32.752583 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d93888a-9672-4e0d-814f-292994fb82de None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:32.775955 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ec49d3-afc4-4d06-a777-846f94c84644 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:32.776633 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ec49d3-afc4-4d06-a777-846f94c84644 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:32.776917 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ec49d3-afc4-4d06-a777-846f94c84644 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:32.777188 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ec49d3-afc4-4d06-a777-846f94c84644 None admin] PATH_INFO: `/v3/users/93596e79352c4e77bb8411f959b7e723` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:32.778391 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ec49d3-afc4-4d06-a777-846f94c84644 None admin] RBAC: Authorizing `identity:get_user(user_id=93596e79352c4e77bb8411f959b7e723)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:32.779851 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ec49d3-afc4-4d06-a777-846f94c84644 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:32.781481 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 80/155] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Nov 2 07:16:32 2022] GET /identity/v3/users/93596e79352c4e77bb8411f959b7e723 => generated 283 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:32.787382 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ec49d3-afc4-4d06-a777-846f94c84644 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:32.797169 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35015890-37ff-4728-95b4-d401dee0c2e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:32.797824 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35015890-37ff-4728-95b4-d401dee0c2e6 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:32.798109 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35015890-37ff-4728-95b4-d401dee0c2e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:32.798415 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35015890-37ff-4728-95b4-d401dee0c2e6 None admin] PATH_INFO: `/v3/projects/826fa0cd5d134fcab8a9411750769a8f` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:32.800250 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35015890-37ff-4728-95b4-d401dee0c2e6 None admin] RBAC: Authorizing `identity:get_project(project_id=826fa0cd5d134fcab8a9411750769a8f)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:32.801891 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35015890-37ff-4728-95b4-d401dee0c2e6 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:32.802766 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:32.804126 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 76/156] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Nov 2 07:16:32 2022] GET /identity/v3/projects/826fa0cd5d134fcab8a9411750769a8f => generated 302 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:32.810696 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35015890-37ff-4728-95b4-d401dee0c2e6 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:32.910078 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ec49d3-afc4-4d06-a777-846f94c84644 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:32.933312 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35015890-37ff-4728-95b4-d401dee0c2e6 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:32.954805 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d639038-81a4-4f7e-ba2a-6e8724b4cf0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:32.955525 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d639038-81a4-4f7e-ba2a-6e8724b4cf0b None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:32.955697 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d639038-81a4-4f7e-ba2a-6e8724b4cf0b None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:32.955955 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d639038-81a4-4f7e-ba2a-6e8724b4cf0b None admin] PATH_INFO: `/v3/projects/826fa0cd5d134fcab8a9411750769a8f` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:32.958051 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d639038-81a4-4f7e-ba2a-6e8724b4cf0b None admin] RBAC: Authorizing `identity:get_project(project_id=826fa0cd5d134fcab8a9411750769a8f)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:32.959515 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d639038-81a4-4f7e-ba2a-6e8724b4cf0b None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:32.961338 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 81/157] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Nov 2 07:16:32 2022] GET /identity/v3/projects/826fa0cd5d134fcab8a9411750769a8f => generated 302 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:32.966969 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d639038-81a4-4f7e-ba2a-6e8724b4cf0b None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:32.972163 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4fb00d-e637-470e-8eb9-99b27dbb6252 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:32.972829 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4fb00d-e637-470e-8eb9-99b27dbb6252 None admin] REQUEST_METHOD: `PUT` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:32.973109 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4fb00d-e637-470e-8eb9-99b27dbb6252 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:32.973430 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4fb00d-e637-470e-8eb9-99b27dbb6252 None admin] PATH_INFO: `/v3/projects/826fa0cd5d134fcab8a9411750769a8f/users/93596e79352c4e77bb8411f959b7e723/roles/c4bf019fc6c44c71a6c984f6d0bab75f` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:32.976372 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4fb00d-e637-470e-8eb9-99b27dbb6252 None admin] RBAC: Authorizing `identity:create_grant(project_id=826fa0cd5d134fcab8a9411750769a8f, user_id=93596e79352c4e77bb8411f959b7e723, role_id=c4bf019fc6c44c71a6c984f6d0bab75f)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:32.977809 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4fb00d-e637-470e-8eb9-99b27dbb6252 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:32.994446 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 77/158] 10.222.0.18 () {58 vars in 1439 bytes} [Wed Nov 2 07:16:32 2022] PUT /identity/v3/projects/826fa0cd5d134fcab8a9411750769a8f/users/93596e79352c4e77bb8411f959b7e723/roles/c4bf019fc6c44c71a6c984f6d0bab75f => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:16:33.066975 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:33.096756 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d639038-81a4-4f7e-ba2a-6e8724b4cf0b None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:33.136560 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8d9bc5-050d-44e9-b3ae-82c49d3f87df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:33.137233 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8d9bc5-050d-44e9-b3ae-82c49d3f87df None admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:33.137511 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8d9bc5-050d-44e9-b3ae-82c49d3f87df None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:33.137782 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8d9bc5-050d-44e9-b3ae-82c49d3f87df None admin] PATH_INFO: `/v3/projects/826fa0cd5d134fcab8a9411750769a8f/users/93596e79352c4e77bb8411f959b7e723/roles/8cf467a4f06c44e0b77499aeababdcc3` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:33.140144 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8d9bc5-050d-44e9-b3ae-82c49d3f87df None admin] RBAC: Authorizing `identity:create_grant(project_id=826fa0cd5d134fcab8a9411750769a8f, user_id=93596e79352c4e77bb8411f959b7e723, role_id=8cf467a4f06c44e0b77499aeababdcc3)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:33.141626 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8d9bc5-050d-44e9-b3ae-82c49d3f87df None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:33.159546 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 82/159] 10.222.0.18 () {58 vars in 1439 bytes} [Wed Nov 2 07:16:32 2022] PUT /identity/v3/projects/826fa0cd5d134fcab8a9411750769a8f/users/93596e79352c4e77bb8411f959b7e723/roles/8cf467a4f06c44e0b77499aeababdcc3 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 07:16:34.951256 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97eedd9-8543-4777-bb9e-ead799f973be None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:34.952275 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97eedd9-8543-4777-bb9e-ead799f973be None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:34.952770 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97eedd9-8543-4777-bb9e-ead799f973be None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:34.954823 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 78/160] 10.222.0.18 () {54 vars in 927 bytes} [Wed Nov 2 07:16:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:16:34.961130 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c1d213-08f9-41b8-b80f-7a6ee45bccef None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:34.962161 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c1d213-08f9-41b8-b80f-7a6ee45bccef None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:34.962713 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c1d213-08f9-41b8-b80f-7a6ee45bccef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:34.984007 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-47c1d213-08f9-41b8-b80f-7a6ee45bccef None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:16:34.985646 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-47c1d213-08f9-41b8-b80f-7a6ee45bccef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:16:35.057966 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 83/161] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Nov 2 07:16:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:16:35.082351 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dd586c-f627-4815-bb99-640746f74e40 None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:35.082656 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dd586c-f627-4815-bb99-640746f74e40 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:35.083189 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dd586c-f627-4815-bb99-640746f74e40 None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:35.084728 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 79/162] 10.222.0.18 () {54 vars in 927 bytes} [Wed Nov 2 07:16:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:16:35.090161 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d32eed-8c5e-4c0c-9635-e43c6a558ddb None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:35.090483 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d32eed-8c5e-4c0c-9635-e43c6a558ddb None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:35.090990 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d32eed-8c5e-4c0c-9635-e43c6a558ddb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:35.111739 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-e9d32eed-8c5e-4c0c-9635-e43c6a558ddb None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:16:35.113448 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-e9d32eed-8c5e-4c0c-9635-e43c6a558ddb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:16:35.188370 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 84/163] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Nov 2 07:16:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:16:35.189115 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:35.244036 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dd4941-be9f-4ac0-8711-9d342f7866d1 None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:35.244309 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dd4941-be9f-4ac0-8711-9d342f7866d1 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:35.244503 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dd4941-be9f-4ac0-8711-9d342f7866d1 None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:35.245983 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 80/164] 10.222.0.18 () {54 vars in 927 bytes} [Wed Nov 2 07:16:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:16:35.251482 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57de1d5-86cb-4c0e-a669-2da119ad43af None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:35.251993 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57de1d5-86cb-4c0e-a669-2da119ad43af None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:35.252573 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57de1d5-86cb-4c0e-a669-2da119ad43af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:35.271362 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-f57de1d5-86cb-4c0e-a669-2da119ad43af None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:16:35.273413 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-f57de1d5-86cb-4c0e-a669-2da119ad43af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:16:35.311807 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d289f9e3-7771-41a0-9aa7-2663c1a644a9 None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:35.312413 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d289f9e3-7771-41a0-9aa7-2663c1a644a9 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:35.312905 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d289f9e3-7771-41a0-9aa7-2663c1a644a9 None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:35.314472 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 81/165] 10.222.0.18 () {54 vars in 927 bytes} [Wed Nov 2 07:16:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:16:35.322602 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a95531-0cf4-4a89-b1e5-2a34de8a53e9 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:35.323256 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a95531-0cf4-4a89-b1e5-2a34de8a53e9 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:35.323599 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a95531-0cf4-4a89-b1e5-2a34de8a53e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:35.343639 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-a3a95531-0cf4-4a89-b1e5-2a34de8a53e9 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:16:35.344831 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-a3a95531-0cf4-4a89-b1e5-2a34de8a53e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:16:35.349296 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 85/166] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Nov 2 07:16:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:16:35.350292 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57de1d5-86cb-4c0e-a669-2da119ad43af None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:35.374202 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:35.421458 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 82/167] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Nov 2 07:16:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Nov 02 07:16:35.469398 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57de1d5-86cb-4c0e-a669-2da119ad43af None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:35.480205 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a95531-0cf4-4a89-b1e5-2a34de8a53e9 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:35.503231 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:35.510990 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ba44b7-6612-4e98-9c32-cd888335e738 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:35.511632 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ba44b7-6612-4e98-9c32-cd888335e738 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:35.511909 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ba44b7-6612-4e98-9c32-cd888335e738 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:35.512329 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ba44b7-6612-4e98-9c32-cd888335e738 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:35.525395 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ba44b7-6612-4e98-9c32-cd888335e738 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:16:35.532180 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-25ba44b7-6612-4e98-9c32-cd888335e738 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Nov 02 07:16:35.532180 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:35.532180 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:16:35.532180 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:16:35.532180 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:16:35.532180 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:16:35.532180 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:16:35.532180 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:16:35.532180 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:16:35.532180 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:16:35.532180 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:16:35.532180 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:16:35.532180 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:16:35.532180 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:16:35.532180 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:16:35.532180 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:16:35.533126 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:16:35.533126 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:16:35.533126 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:16:35.533126 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:16:35.533126 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:16:35.533126 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:16:35.533126 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:16:35.533126 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:16:35.533126 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:16:35.533126 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:16:35.533126 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:16:35.533126 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:16:35.533126 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:16:35.533126 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:16:35.533126 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:16:35.533126 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:16:35.533887 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:16:35.533887 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:16:35.533887 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:16:35.533887 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:16:35.533887 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:16:35.533887 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:16:35.533887 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:16:35.533887 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:16:35.533887 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:16:35.533887 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:16:35.533887 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Nov 02 07:16:35.533887 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:16:35.533887 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 86/168] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Nov 2 07:16:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:16:35.539255 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ba44b7-6612-4e98-9c32-cd888335e738 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:35.588835 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a95531-0cf4-4a89-b1e5-2a34de8a53e9 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:35.626558 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4a8377-fdc0-4732-923f-12ab30693eda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:35.627320 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4a8377-fdc0-4732-923f-12ab30693eda None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:35.627758 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4a8377-fdc0-4732-923f-12ab30693eda None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:35.628096 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4a8377-fdc0-4732-923f-12ab30693eda None admin] PATH_INFO: `/v3/domains/default` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:35.629198 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4a8377-fdc0-4732-923f-12ab30693eda None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:35.630922 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4a8377-fdc0-4732-923f-12ab30693eda None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:35.632761 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 83/169] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Nov 2 07:16:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:16:35.638056 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4a8377-fdc0-4732-923f-12ab30693eda None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:35.639642 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ba44b7-6612-4e98-9c32-cd888335e738 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:35.666975 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:35.683931 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82058f8-ae56-4945-852b-b0aaf5985cb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:35.684628 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82058f8-ae56-4945-852b-b0aaf5985cb1 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:35.685263 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82058f8-ae56-4945-852b-b0aaf5985cb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:35.685543 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82058f8-ae56-4945-852b-b0aaf5985cb1 None admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:35.686203 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82058f8-ae56-4945-852b-b0aaf5985cb1 None admin] RBAC: Adding query filter params (name=member) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:35.686842 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82058f8-ae56-4945-852b-b0aaf5985cb1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:35.688204 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82058f8-ae56-4945-852b-b0aaf5985cb1 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:35.696777 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 87/170] 10.222.0.18 () {56 vars in 1101 bytes} [Wed Nov 2 07:16:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:16:35.702229 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82058f8-ae56-4945-852b-b0aaf5985cb1 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:35.743012 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4a8377-fdc0-4732-923f-12ab30693eda None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:35.754048 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:35.777058 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29d995b-7af8-479a-8e40-10c666749688 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:35.777689 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29d995b-7af8-479a-8e40-10c666749688 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:35.777972 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29d995b-7af8-479a-8e40-10c666749688 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:35.778242 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29d995b-7af8-479a-8e40-10c666749688 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:35.788207 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29d995b-7af8-479a-8e40-10c666749688 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:16:35.793081 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-c29d995b-7af8-479a-8e40-10c666749688 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 07:16:35.793081 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:35.793081 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:16:35.793081 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:16:35.793081 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:16:35.793081 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:16:35.793081 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:16:35.793081 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:16:35.793081 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:16:35.793081 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:16:35.793081 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:16:35.793081 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:16:35.793081 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:16:35.793081 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:16:35.793081 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:16:35.793081 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:16:35.793986 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:16:35.793986 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:16:35.793986 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:16:35.793986 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:16:35.793986 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:16:35.793986 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:16:35.793986 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:16:35.793986 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:16:35.793986 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:16:35.793986 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:16:35.793986 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:16:35.793986 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Nov 02 07:16:35.793986 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:16:35.793986 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:16:35.793986 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:16:35.793986 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:16:35.794776 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:16:35.794776 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:16:35.794776 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:16:35.794776 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:16:35.794776 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:16:35.794776 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:16:35.794776 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:16:35.794776 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:16:35.794776 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:16:35.794776 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:16:35.794776 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 07:16:35.794776 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Nov 02 07:16:35.794776 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 84/171] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Nov 2 07:16:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:16:35.795424 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29d995b-7af8-479a-8e40-10c666749688 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:35.806558 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82058f8-ae56-4945-852b-b0aaf5985cb1 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:35.839531 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00d6c5e-5807-4441-b154-cbf58f48b164 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:35.840136 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00d6c5e-5807-4441-b154-cbf58f48b164 None admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:35.840411 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00d6c5e-5807-4441-b154-cbf58f48b164 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:35.840679 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00d6c5e-5807-4441-b154-cbf58f48b164 None admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:35.841321 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00d6c5e-5807-4441-b154-cbf58f48b164 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:35.842812 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00d6c5e-5807-4441-b154-cbf58f48b164 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:35.868791 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 88/172] 10.222.0.18 () {60 vars in 1132 bytes} [Wed Nov 2 07:16:35 2022] POST /identity/v3/users => generated 297 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:16:35.869615 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00d6c5e-5807-4441-b154-cbf58f48b164 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:35.907067 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:35.934564 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29d995b-7af8-479a-8e40-10c666749688 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:35.977178 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59833903-d19a-4e86-bfc0-d92a9119c2aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:35.977868 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59833903-d19a-4e86-bfc0-d92a9119c2aa None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:35.978159 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59833903-d19a-4e86-bfc0-d92a9119c2aa None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:35.978391 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59833903-d19a-4e86-bfc0-d92a9119c2aa None admin] PATH_INFO: `/v3/users/987eaf41604549d08df48f1acc7e057f` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:35.995026 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59833903-d19a-4e86-bfc0-d92a9119c2aa None admin] RBAC: Authorizing `identity:get_user(user_id=987eaf41604549d08df48f1acc7e057f)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:35.996564 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59833903-d19a-4e86-bfc0-d92a9119c2aa None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:35.998497 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 85/173] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Nov 2 07:16:35 2022] GET /identity/v3/users/987eaf41604549d08df48f1acc7e057f => generated 297 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:35.999118 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59833903-d19a-4e86-bfc0-d92a9119c2aa None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:36.018247 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00d6c5e-5807-4441-b154-cbf58f48b164 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:36.029983 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:36.062923 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e83830-8779-417a-bae7-29979b64ceec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:36.063557 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e83830-8779-417a-bae7-29979b64ceec None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:36.063836 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e83830-8779-417a-bae7-29979b64ceec None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:36.064109 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e83830-8779-417a-bae7-29979b64ceec None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:36.075319 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e83830-8779-417a-bae7-29979b64ceec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:16:36.080604 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-81e83830-8779-417a-bae7-29979b64ceec None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Nov 02 07:16:36.080604 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:36.080604 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:16:36.080604 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:16:36.080604 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:16:36.080604 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:16:36.080604 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:16:36.080604 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:16:36.080604 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:16:36.080604 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:16:36.080604 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:16:36.080604 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:16:36.080604 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:16:36.080604 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:16:36.080604 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:16:36.080604 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:16:36.081683 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:16:36.081683 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:16:36.081683 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:16:36.081683 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:16:36.081683 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:16:36.081683 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:16:36.081683 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:16:36.081683 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:16:36.081683 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:16:36.081683 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:16:36.081683 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:16:36.081683 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:16:36.081683 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:16:36.081683 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:16:36.081683 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:16:36.081683 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:16:36.082660 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:16:36.082660 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:16:36.082660 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:16:36.082660 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:16:36.082660 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:16:36.082660 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:16:36.082660 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:16:36.082660 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:16:36.082660 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:16:36.082660 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:16:36.082660 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Nov 02 07:16:36.082660 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:16:36.082660 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 89/174] 10.222.0.18 () {56 vars in 1114 bytes} [Wed Nov 2 07:16:35 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:16:36.082660 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e83830-8779-417a-bae7-29979b64ceec None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:36.126523 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59833903-d19a-4e86-bfc0-d92a9119c2aa None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:36.171487 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc92a3a-00b3-450e-91b8-eb5e41157366 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:36.172104 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc92a3a-00b3-450e-91b8-eb5e41157366 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:36.172384 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc92a3a-00b3-450e-91b8-eb5e41157366 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:36.172659 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc92a3a-00b3-450e-91b8-eb5e41157366 None admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:36.173331 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc92a3a-00b3-450e-91b8-eb5e41157366 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:36.173635 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc92a3a-00b3-450e-91b8-eb5e41157366 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:36.175059 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc92a3a-00b3-450e-91b8-eb5e41157366 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:36.188107 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 86/175] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Nov 2 07:16:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:36.189045 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc92a3a-00b3-450e-91b8-eb5e41157366 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:36.200099 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:36.200488 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e83830-8779-417a-bae7-29979b64ceec None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:36.254690 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a10884c-d87a-4490-bbfe-3c92a0ac7d3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:36.255333 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a10884c-d87a-4490-bbfe-3c92a0ac7d3d None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:36.255614 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a10884c-d87a-4490-bbfe-3c92a0ac7d3d None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:36.255928 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a10884c-d87a-4490-bbfe-3c92a0ac7d3d None admin] PATH_INFO: `/v3/projects/826fa0cd5d134fcab8a9411750769a8f` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:36.258501 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a10884c-d87a-4490-bbfe-3c92a0ac7d3d None admin] RBAC: Authorizing `identity:get_project(project_id=826fa0cd5d134fcab8a9411750769a8f)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:36.260406 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a10884c-d87a-4490-bbfe-3c92a0ac7d3d None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:36.262981 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 90/176] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Nov 2 07:16:36 2022] GET /identity/v3/projects/826fa0cd5d134fcab8a9411750769a8f => generated 302 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:36.263782 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a10884c-d87a-4490-bbfe-3c92a0ac7d3d None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:36.313543 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:36.316169 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc92a3a-00b3-450e-91b8-eb5e41157366 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:36.355046 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559acb4b-01ef-46d8-ac60-569f8fe93779 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:36.355690 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559acb4b-01ef-46d8-ac60-569f8fe93779 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:36.355973 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559acb4b-01ef-46d8-ac60-569f8fe93779 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:36.356245 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559acb4b-01ef-46d8-ac60-569f8fe93779 None admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:36.356916 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559acb4b-01ef-46d8-ac60-569f8fe93779 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:36.357218 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559acb4b-01ef-46d8-ac60-569f8fe93779 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:36.358631 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559acb4b-01ef-46d8-ac60-569f8fe93779 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:36.368570 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 87/177] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Nov 2 07:16:36 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:36.369350 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559acb4b-01ef-46d8-ac60-569f8fe93779 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:36.401817 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a10884c-d87a-4490-bbfe-3c92a0ac7d3d None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:36.437547 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9d226e-0bcd-4fea-8424-8123ca7f442c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:36.438627 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9d226e-0bcd-4fea-8424-8123ca7f442c None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:36.439114 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9d226e-0bcd-4fea-8424-8123ca7f442c None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:36.439587 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9d226e-0bcd-4fea-8424-8123ca7f442c None admin] PATH_INFO: `/v3/users/93596e79352c4e77bb8411f959b7e723` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:36.440946 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9d226e-0bcd-4fea-8424-8123ca7f442c None admin] RBAC: Authorizing `identity:get_user(user_id=93596e79352c4e77bb8411f959b7e723)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:36.441707 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:36.442943 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9d226e-0bcd-4fea-8424-8123ca7f442c None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:36.445383 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 91/178] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Nov 2 07:16:36 2022] GET /identity/v3/users/93596e79352c4e77bb8411f959b7e723 => generated 283 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:36.450803 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9d226e-0bcd-4fea-8424-8123ca7f442c None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:36.481316 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559acb4b-01ef-46d8-ac60-569f8fe93779 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:36.519828 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85f847e-a996-45d9-8557-d57ca4672bae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:36.520496 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85f847e-a996-45d9-8557-d57ca4672bae None admin] REQUEST_METHOD: `PUT` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:36.520779 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85f847e-a996-45d9-8557-d57ca4672bae None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:36.521049 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85f847e-a996-45d9-8557-d57ca4672bae None admin] PATH_INFO: `/v3/projects/826fa0cd5d134fcab8a9411750769a8f/users/987eaf41604549d08df48f1acc7e057f/roles/b494fedd00f24f0b84a529e8672a156a` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:36.523252 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85f847e-a996-45d9-8557-d57ca4672bae None admin] RBAC: Authorizing `identity:create_grant(project_id=826fa0cd5d134fcab8a9411750769a8f, user_id=987eaf41604549d08df48f1acc7e057f, role_id=b494fedd00f24f0b84a529e8672a156a)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:36.524709 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85f847e-a996-45d9-8557-d57ca4672bae None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:36.542809 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 88/179] 10.222.0.18 () {58 vars in 1439 bytes} [Wed Nov 2 07:16:36 2022] PUT /identity/v3/projects/826fa0cd5d134fcab8a9411750769a8f/users/987eaf41604549d08df48f1acc7e057f/roles/b494fedd00f24f0b84a529e8672a156a => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:16:36.551151 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85f847e-a996-45d9-8557-d57ca4672bae None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:36.600760 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9d226e-0bcd-4fea-8424-8123ca7f442c None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:36.611728 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:36.653762 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb536bf-808b-4dcd-aa99-e9dde6733385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:36.654836 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb536bf-808b-4dcd-aa99-e9dde6733385 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:36.655356 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb536bf-808b-4dcd-aa99-e9dde6733385 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:36.655886 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb536bf-808b-4dcd-aa99-e9dde6733385 None admin] PATH_INFO: `/v3/users/93596e79352c4e77bb8411f959b7e723` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:36.657463 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb536bf-808b-4dcd-aa99-e9dde6733385 None admin] RBAC: Authorizing `identity:get_user(user_id=93596e79352c4e77bb8411f959b7e723)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:36.659286 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb536bf-808b-4dcd-aa99-e9dde6733385 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:36.661539 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 92/180] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Nov 2 07:16:36 2022] GET /identity/v3/users/93596e79352c4e77bb8411f959b7e723 => generated 283 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:36.668333 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb536bf-808b-4dcd-aa99-e9dde6733385 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:36.690631 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85f847e-a996-45d9-8557-d57ca4672bae None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:36.725836 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:36.741033 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f444d1e-04d7-43c4-befe-779dd51ae160 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:36.742440 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f444d1e-04d7-43c4-befe-779dd51ae160 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:36.743007 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f444d1e-04d7-43c4-befe-779dd51ae160 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:36.743494 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f444d1e-04d7-43c4-befe-779dd51ae160 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:36.841705 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f444d1e-04d7-43c4-befe-779dd51ae160 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:36.843213 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f444d1e-04d7-43c4-befe-779dd51ae160 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:36.882069 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb536bf-808b-4dcd-aa99-e9dde6733385 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:36.941190 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9f3f4c-58a7-403c-a950-2acef7f7e104 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:36.941823 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9f3f4c-58a7-403c-a950-2acef7f7e104 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:36.942102 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9f3f4c-58a7-403c-a950-2acef7f7e104 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:36.942384 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9f3f4c-58a7-403c-a950-2acef7f7e104 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:37.026578 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:37.058581 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9f3f4c-58a7-403c-a950-2acef7f7e104 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:37.060063 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9f3f4c-58a7-403c-a950-2acef7f7e104 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:37.073463 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 89/181] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Nov 2 07:16:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 530 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Nov 02 07:16:37.079247 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f444d1e-04d7-43c4-befe-779dd51ae160 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:37.135537 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:37.191831 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 93/182] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Nov 2 07:16:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 524 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Nov 02 07:16:37.198666 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9f3f4c-58a7-403c-a950-2acef7f7e104 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:37.268514 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f444d1e-04d7-43c4-befe-779dd51ae160 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:37.297805 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:37.317096 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee3ab03-b9ac-4b6a-8ac1-1b8d230f82c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:37.317096 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee3ab03-b9ac-4b6a-8ac1-1b8d230f82c8 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:37.317096 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee3ab03-b9ac-4b6a-8ac1-1b8d230f82c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:37.317096 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee3ab03-b9ac-4b6a-8ac1-1b8d230f82c8 None admin] PATH_INFO: `/v3/projects/826fa0cd5d134fcab8a9411750769a8f` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:37.317496 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee3ab03-b9ac-4b6a-8ac1-1b8d230f82c8 None admin] RBAC: Authorizing `identity:get_project(project_id=826fa0cd5d134fcab8a9411750769a8f)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:37.317496 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee3ab03-b9ac-4b6a-8ac1-1b8d230f82c8 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:37.324640 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 90/183] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Nov 2 07:16:37 2022] GET /identity/v3/projects/826fa0cd5d134fcab8a9411750769a8f => generated 302 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:16:37.325482 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee3ab03-b9ac-4b6a-8ac1-1b8d230f82c8 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:37.349609 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9f3f4c-58a7-403c-a950-2acef7f7e104 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:37.453013 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:37.456051 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3a375b-a442-47fc-86bd-85b5d865afa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:37.456701 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3a375b-a442-47fc-86bd-85b5d865afa2 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:37.457113 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3a375b-a442-47fc-86bd-85b5d865afa2 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:37.457383 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3a375b-a442-47fc-86bd-85b5d865afa2 None admin] PATH_INFO: `/v3/projects/826fa0cd5d134fcab8a9411750769a8f` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:37.458768 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3a375b-a442-47fc-86bd-85b5d865afa2 None admin] RBAC: Authorizing `identity:get_project(project_id=826fa0cd5d134fcab8a9411750769a8f)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:37.459730 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee3ab03-b9ac-4b6a-8ac1-1b8d230f82c8 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:37.461954 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3a375b-a442-47fc-86bd-85b5d865afa2 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:37.463917 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 94/184] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Nov 2 07:16:37 2022] GET /identity/v3/projects/826fa0cd5d134fcab8a9411750769a8f => generated 302 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:37.471998 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3a375b-a442-47fc-86bd-85b5d865afa2 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:37.511060 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d622dbf-6459-4339-9bed-bcc235957581 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:37.511667 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d622dbf-6459-4339-9bed-bcc235957581 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:37.511942 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d622dbf-6459-4339-9bed-bcc235957581 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:37.512209 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d622dbf-6459-4339-9bed-bcc235957581 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:37.512928 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d622dbf-6459-4339-9bed-bcc235957581 None admin] RBAC: Adding query filter params (role.id=c4bf019fc6c44c71a6c984f6d0bab75f, scope.project.id=826fa0cd5d134fcab8a9411750769a8f, user.id=93596e79352c4e77bb8411f959b7e723) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:37.513230 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d622dbf-6459-4339-9bed-bcc235957581 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:37.514675 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d622dbf-6459-4339-9bed-bcc235957581 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:37.529547 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 91/185] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Nov 2 07:16:37 2022] GET /identity/v3/role_assignments?user.id=93596e79352c4e77bb8411f959b7e723&scope.project.id=826fa0cd5d134fcab8a9411750769a8f&role.id=c4bf019fc6c44c71a6c984f6d0bab75f => generated 608 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:37.665224 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3a375b-a442-47fc-86bd-85b5d865afa2 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:37.714894 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c05546-bbeb-4047-b532-d4d8833b86e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:37.715478 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c05546-bbeb-4047-b532-d4d8833b86e6 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:37.715756 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c05546-bbeb-4047-b532-d4d8833b86e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:37.716031 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c05546-bbeb-4047-b532-d4d8833b86e6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:37.716761 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c05546-bbeb-4047-b532-d4d8833b86e6 None admin] RBAC: Adding query filter params (role.id=8cf467a4f06c44e0b77499aeababdcc3, scope.project.id=826fa0cd5d134fcab8a9411750769a8f, user.id=93596e79352c4e77bb8411f959b7e723) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:37.717067 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c05546-bbeb-4047-b532-d4d8833b86e6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:37.718537 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c05546-bbeb-4047-b532-d4d8833b86e6 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:37.724803 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 95/186] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Nov 2 07:16:37 2022] GET /identity/v3/role_assignments?user.id=93596e79352c4e77bb8411f959b7e723&scope.project.id=826fa0cd5d134fcab8a9411750769a8f&role.id=8cf467a4f06c44e0b77499aeababdcc3 => generated 608 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:38.429131 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf295f9-d3b9-4a74-9278-073746725231 None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:38.429652 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf295f9-d3b9-4a74-9278-073746725231 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:38.429652 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf295f9-d3b9-4a74-9278-073746725231 None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:38.431359 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 92/187] 10.222.0.18 () {54 vars in 927 bytes} [Wed Nov 2 07:16:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:16:38.436310 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bc9954-c0d9-40b0-9c22-9c813e0b5d8b None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:38.436694 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bc9954-c0d9-40b0-9c22-9c813e0b5d8b None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:38.436874 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bc9954-c0d9-40b0-9c22-9c813e0b5d8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:38.444071 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:38.455354 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-f1bc9954-c0d9-40b0-9c22-9c813e0b5d8b None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:16:38.456524 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-f1bc9954-c0d9-40b0-9c22-9c813e0b5d8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:16:38.512358 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b399971-413f-4b63-8916-95701911158b None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:38.512739 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b399971-413f-4b63-8916-95701911158b None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:38.512911 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b399971-413f-4b63-8916-95701911158b None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:38.514245 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 93/188] 10.222.0.18 () {54 vars in 927 bytes} [Wed Nov 2 07:16:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Nov 02 07:16:38.519024 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bbe78e-0bee-4ab5-86b5-aa49f1399506 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:38.519307 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bbe78e-0bee-4ab5-86b5-aa49f1399506 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:38.519577 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bbe78e-0bee-4ab5-86b5-aa49f1399506 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:38.524811 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 96/189] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Nov 2 07:16:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:16:38.538827 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-b7bbe78e-0bee-4ab5-86b5-aa49f1399506 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:16:38.540242 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-b7bbe78e-0bee-4ab5-86b5-aa49f1399506 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:16:38.605900 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 94/190] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Nov 2 07:16:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Nov 02 07:16:38.808782 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bc9954-c0d9-40b0-9c22-9c813e0b5d8b None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:38.898712 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bbe78e-0bee-4ab5-86b5-aa49f1399506 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:38.934264 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bc9954-c0d9-40b0-9c22-9c813e0b5d8b None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:38.974863 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531acd60-3ee9-4348-8847-a210fa760d88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:38.975504 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531acd60-3ee9-4348-8847-a210fa760d88 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:38.975810 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531acd60-3ee9-4348-8847-a210fa760d88 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:38.976082 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531acd60-3ee9-4348-8847-a210fa760d88 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:38.977938 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:38.989977 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531acd60-3ee9-4348-8847-a210fa760d88 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:16:38.995419 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-531acd60-3ee9-4348-8847-a210fa760d88 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Nov 02 07:16:38.995419 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:38.995419 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:16:38.995419 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:16:38.995419 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:16:38.995419 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:16:38.995419 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:16:38.995419 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:16:38.995419 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:16:38.995419 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:16:38.995419 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:16:38.995419 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:16:38.995419 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:16:38.995419 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:16:38.995419 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:16:38.995419 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:16:38.996543 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:16:38.996543 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:16:38.996543 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:16:38.996543 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:16:38.996543 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:16:38.996543 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:16:38.996543 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:16:38.996543 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:16:38.996543 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:16:38.996543 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:16:38.996543 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:16:38.996543 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:16:38.996543 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:16:38.996543 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:16:38.996543 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:16:38.996543 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:16:38.997331 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:16:38.997331 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:16:38.997331 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:16:38.997331 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:16:38.997331 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:16:38.997331 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:16:38.997331 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:16:38.997331 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:16:38.997331 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:16:38.997331 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:16:38.997331 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Nov 02 07:16:38.997331 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:16:38.997331 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 97/191] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Nov 2 07:16:38 2022] GET /identity/v3/roles/reader => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:16:38.997982 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11df5a7f-b5c5-44ea-9f4b-a9d1b25058aa None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:38.998551 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11df5a7f-b5c5-44ea-9f4b-a9d1b25058aa None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:38.998551 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11df5a7f-b5c5-44ea-9f4b-a9d1b25058aa None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:39.000412 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 98/192] 10.222.0.18 () {54 vars in 927 bytes} [Wed Nov 2 07:16:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:16:39.003009 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11df5a7f-b5c5-44ea-9f4b-a9d1b25058aa None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:39.005438 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:39.020859 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bbe78e-0bee-4ab5-86b5-aa49f1399506 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:39.060566 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a3e81a-caea-4c2f-9b89-c52357e9be09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:39.061129 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a3e81a-caea-4c2f-9b89-c52357e9be09 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:39.061396 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a3e81a-caea-4c2f-9b89-c52357e9be09 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:39.061688 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a3e81a-caea-4c2f-9b89-c52357e9be09 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:39.062980 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a3e81a-caea-4c2f-9b89-c52357e9be09 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:39.064275 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a3e81a-caea-4c2f-9b89-c52357e9be09 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:39.065852 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 95/193] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Nov 2 07:16:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:16:39.067146 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b354fdc2-7a93-4bf4-90ac-5ce16847bc37 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:39.067432 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b354fdc2-7a93-4bf4-90ac-5ce16847bc37 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:39.067699 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b354fdc2-7a93-4bf4-90ac-5ce16847bc37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:39.091692 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-b354fdc2-7a93-4bf4-90ac-5ce16847bc37 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:16:39.093126 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-b354fdc2-7a93-4bf4-90ac-5ce16847bc37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:16:39.122932 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11df5a7f-b5c5-44ea-9f4b-a9d1b25058aa None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:39.163565 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e87d3b-756c-4e4a-b44a-59fdef5ac1ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:39.164445 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e87d3b-756c-4e4a-b44a-59fdef5ac1ff None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:39.164724 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e87d3b-756c-4e4a-b44a-59fdef5ac1ff None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:39.164991 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e87d3b-756c-4e4a-b44a-59fdef5ac1ff None admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:39.165663 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e87d3b-756c-4e4a-b44a-59fdef5ac1ff None admin] RBAC: Adding query filter params (name=reader) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:39.165957 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e87d3b-756c-4e4a-b44a-59fdef5ac1ff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:39.167239 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 96/194] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Nov 2 07:16:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:16:39.167356 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e87d3b-756c-4e4a-b44a-59fdef5ac1ff None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:39.168060 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b354fdc2-7a93-4bf4-90ac-5ce16847bc37 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:39.180505 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 99/195] 10.222.0.18 () {56 vars in 1101 bytes} [Wed Nov 2 07:16:39 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:16:39.186626 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e87d3b-756c-4e4a-b44a-59fdef5ac1ff None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:39.258694 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:39.274714 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:39.288242 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b354fdc2-7a93-4bf4-90ac-5ce16847bc37 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:39.314836 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e87d3b-756c-4e4a-b44a-59fdef5ac1ff None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:39.333570 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8469d7df-2e3c-4562-81c1-ea8de48b3d66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:39.334188 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8469d7df-2e3c-4562-81c1-ea8de48b3d66 None admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:39.334504 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8469d7df-2e3c-4562-81c1-ea8de48b3d66 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:39.334782 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8469d7df-2e3c-4562-81c1-ea8de48b3d66 None admin] PATH_INFO: `/v3/users` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:39.336116 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8469d7df-2e3c-4562-81c1-ea8de48b3d66 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:39.338115 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8469d7df-2e3c-4562-81c1-ea8de48b3d66 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:39.355463 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487a8799-f5ad-46a1-a0bb-dd273264cbb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:39.356066 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487a8799-f5ad-46a1-a0bb-dd273264cbb2 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:39.356366 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487a8799-f5ad-46a1-a0bb-dd273264cbb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:39.356697 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487a8799-f5ad-46a1-a0bb-dd273264cbb2 None admin] PATH_INFO: `/v3/users/4c414e37665448af88c06472b37e608c` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:39.368396 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 97/196] 10.222.0.18 () {60 vars in 1132 bytes} [Wed Nov 2 07:16:39 2022] POST /identity/v3/users => generated 293 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:16:39.386123 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487a8799-f5ad-46a1-a0bb-dd273264cbb2 None admin] RBAC: Authorizing `identity:get_user(user_id=4c414e37665448af88c06472b37e608c)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:39.387706 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487a8799-f5ad-46a1-a0bb-dd273264cbb2 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:39.389742 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 100/197] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Nov 2 07:16:39 2022] GET /identity/v3/users/4c414e37665448af88c06472b37e608c => generated 297 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:39.397120 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8469d7df-2e3c-4562-81c1-ea8de48b3d66 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:39.472581 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487a8799-f5ad-46a1-a0bb-dd273264cbb2 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:39.526173 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8469d7df-2e3c-4562-81c1-ea8de48b3d66 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:39.569542 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79612a8-1c63-422b-b943-5ae4e20142b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:39.570184 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79612a8-1c63-422b-b943-5ae4e20142b7 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:39.570412 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79612a8-1c63-422b-b943-5ae4e20142b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:39.570671 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79612a8-1c63-422b-b943-5ae4e20142b7 None admin] PATH_INFO: `/v3/projects/826fa0cd5d134fcab8a9411750769a8f` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:39.571899 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79612a8-1c63-422b-b943-5ae4e20142b7 None admin] RBAC: Authorizing `identity:get_project(project_id=826fa0cd5d134fcab8a9411750769a8f)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:39.573354 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79612a8-1c63-422b-b943-5ae4e20142b7 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:39.575331 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 98/198] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Nov 2 07:16:39 2022] GET /identity/v3/projects/826fa0cd5d134fcab8a9411750769a8f => generated 302 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:39.580601 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79612a8-1c63-422b-b943-5ae4e20142b7 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:39.597063 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:39.601369 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487a8799-f5ad-46a1-a0bb-dd273264cbb2 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:39.619133 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:39.651043 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dad8faf-4cb8-4119-a47f-520a744f9a27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:39.651691 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dad8faf-4cb8-4119-a47f-520a744f9a27 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:39.651968 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dad8faf-4cb8-4119-a47f-520a744f9a27 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:39.652236 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dad8faf-4cb8-4119-a47f-520a744f9a27 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:39.666091 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dad8faf-4cb8-4119-a47f-520a744f9a27 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:16:39.671427 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-9dad8faf-4cb8-4119-a47f-520a744f9a27 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Nov 02 07:16:39.671427 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:39.671427 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:16:39.671427 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:16:39.671427 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:16:39.671427 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:16:39.671427 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:16:39.671427 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:16:39.671427 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:16:39.671427 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:16:39.671427 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:16:39.671427 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:16:39.671427 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:16:39.671427 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:16:39.671427 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:16:39.671427 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:16:39.672287 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:16:39.672287 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:16:39.672287 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:16:39.672287 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:16:39.672287 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:16:39.672287 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:16:39.672287 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:16:39.672287 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:16:39.672287 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:16:39.672287 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:16:39.672287 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:16:39.672287 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:16:39.672287 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:16:39.672287 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:16:39.672287 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:16:39.672287 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:16:39.673087 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:16:39.673087 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:16:39.673087 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:16:39.673087 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:16:39.673087 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:16:39.673087 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:16:39.673087 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:16:39.673087 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:16:39.673087 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:16:39.673087 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:16:39.673087 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Nov 02 07:16:39.673087 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:16:39.675431 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 101/199] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Nov 2 07:16:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:16:39.682343 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dad8faf-4cb8-4119-a47f-520a744f9a27 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:39.706477 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79612a8-1c63-422b-b943-5ae4e20142b7 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:39.753844 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11982012-d61d-4873-8285-557536f78513 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:39.755448 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11982012-d61d-4873-8285-557536f78513 None admin] REQUEST_METHOD: `PUT` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:39.755448 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11982012-d61d-4873-8285-557536f78513 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:39.755448 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11982012-d61d-4873-8285-557536f78513 None admin] PATH_INFO: `/v3/projects/826fa0cd5d134fcab8a9411750769a8f/users/4c414e37665448af88c06472b37e608c/roles/979bdb3f4d8d4d97b1729d3162372a09` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:39.757434 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11982012-d61d-4873-8285-557536f78513 None admin] RBAC: Authorizing `identity:create_grant(project_id=826fa0cd5d134fcab8a9411750769a8f, user_id=4c414e37665448af88c06472b37e608c, role_id=979bdb3f4d8d4d97b1729d3162372a09)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:39.758934 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11982012-d61d-4873-8285-557536f78513 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:39.774144 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 99/200] 10.222.0.18 () {58 vars in 1439 bytes} [Wed Nov 2 07:16:39 2022] PUT /identity/v3/projects/826fa0cd5d134fcab8a9411750769a8f/users/4c414e37665448af88c06472b37e608c/roles/979bdb3f4d8d4d97b1729d3162372a09 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 07:16:39.825882 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dad8faf-4cb8-4119-a47f-520a744f9a27 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:39.868365 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c6cd5c-921d-4b62-a1d3-a2c0c70ec1c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:39.869460 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c6cd5c-921d-4b62-a1d3-a2c0c70ec1c2 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:39.870044 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c6cd5c-921d-4b62-a1d3-a2c0c70ec1c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:39.870554 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c6cd5c-921d-4b62-a1d3-a2c0c70ec1c2 None admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:39.871481 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c6cd5c-921d-4b62-a1d3-a2c0c70ec1c2 None admin] RBAC: Adding query filter params (name=member) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:39.872111 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c6cd5c-921d-4b62-a1d3-a2c0c70ec1c2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:39.873792 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c6cd5c-921d-4b62-a1d3-a2c0c70ec1c2 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:39.885192 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 102/201] 10.222.0.18 () {56 vars in 1101 bytes} [Wed Nov 2 07:16:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:16:39.891017 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11982012-d61d-4873-8285-557536f78513 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:39.973654 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:40.010619 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11982012-d61d-4873-8285-557536f78513 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:40.046061 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850aec3f-c684-4e22-9426-d8345e0f5624 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:40.047133 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850aec3f-c684-4e22-9426-d8345e0f5624 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:40.047617 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850aec3f-c684-4e22-9426-d8345e0f5624 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:40.048088 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850aec3f-c684-4e22-9426-d8345e0f5624 None admin] PATH_INFO: `/v3/users/987eaf41604549d08df48f1acc7e057f` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:40.049422 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850aec3f-c684-4e22-9426-d8345e0f5624 None admin] RBAC: Authorizing `identity:get_user(user_id=987eaf41604549d08df48f1acc7e057f)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:40.051142 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850aec3f-c684-4e22-9426-d8345e0f5624 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:40.052966 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 100/202] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Nov 2 07:16:39 2022] GET /identity/v3/users/987eaf41604549d08df48f1acc7e057f => generated 297 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:16:40.059271 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c6cd5c-921d-4b62-a1d3-a2c0c70ec1c2 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:40.096346 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:40.163757 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c6cd5c-921d-4b62-a1d3-a2c0c70ec1c2 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:40.201760 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f18538f-f02f-4b28-a942-ef0d8dac3ce5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:40.202457 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f18538f-f02f-4b28-a942-ef0d8dac3ce5 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:40.202736 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f18538f-f02f-4b28-a942-ef0d8dac3ce5 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:40.203005 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f18538f-f02f-4b28-a942-ef0d8dac3ce5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:40.272997 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f18538f-f02f-4b28-a942-ef0d8dac3ce5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:40.274490 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f18538f-f02f-4b28-a942-ef0d8dac3ce5 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:40.310330 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:40.386904 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 103/203] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Nov 2 07:16:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Nov 02 07:16:40.398437 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850aec3f-c684-4e22-9426-d8345e0f5624 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:40.500517 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850aec3f-c684-4e22-9426-d8345e0f5624 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:40.534736 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3707d3-0107-4882-a11d-bba70a50eb26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:40.535387 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3707d3-0107-4882-a11d-bba70a50eb26 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:40.535626 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3707d3-0107-4882-a11d-bba70a50eb26 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:40.535898 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3707d3-0107-4882-a11d-bba70a50eb26 None admin] PATH_INFO: `/v3/projects/826fa0cd5d134fcab8a9411750769a8f` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:40.537013 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3707d3-0107-4882-a11d-bba70a50eb26 None admin] RBAC: Authorizing `identity:get_project(project_id=826fa0cd5d134fcab8a9411750769a8f)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:40.538553 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3707d3-0107-4882-a11d-bba70a50eb26 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:40.540508 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 101/204] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Nov 2 07:16:40 2022] GET /identity/v3/projects/826fa0cd5d134fcab8a9411750769a8f => generated 302 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:40.546498 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f18538f-f02f-4b28-a942-ef0d8dac3ce5 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:40.656345 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f18538f-f02f-4b28-a942-ef0d8dac3ce5 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:40.692962 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa29b6c3-2fd4-424c-bd02-f171e7e90dea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:40.693570 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa29b6c3-2fd4-424c-bd02-f171e7e90dea None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:40.693851 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa29b6c3-2fd4-424c-bd02-f171e7e90dea None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:40.694121 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa29b6c3-2fd4-424c-bd02-f171e7e90dea None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:40.694903 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa29b6c3-2fd4-424c-bd02-f171e7e90dea None admin] RBAC: Adding query filter params (role.id=b494fedd00f24f0b84a529e8672a156a, scope.project.id=826fa0cd5d134fcab8a9411750769a8f, user.id=987eaf41604549d08df48f1acc7e057f) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:40.695206 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa29b6c3-2fd4-424c-bd02-f171e7e90dea None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:40.696635 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa29b6c3-2fd4-424c-bd02-f171e7e90dea None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:40.699504 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:40.703878 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 104/205] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Nov 2 07:16:40 2022] GET /identity/v3/role_assignments?user.id=987eaf41604549d08df48f1acc7e057f&scope.project.id=826fa0cd5d134fcab8a9411750769a8f&role.id=b494fedd00f24f0b84a529e8672a156a => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:41.774665 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63da6c1-2d9c-49a3-aab8-69858fcf7707 None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:41.775231 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63da6c1-2d9c-49a3-aab8-69858fcf7707 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:41.775231 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63da6c1-2d9c-49a3-aab8-69858fcf7707 None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:41.776796 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 102/206] 10.222.0.18 () {54 vars in 927 bytes} [Wed Nov 2 07:16:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Nov 02 07:16:41.782463 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176cda27-b42d-48c4-b9ea-5e36f8f5f62f None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:41.782957 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176cda27-b42d-48c4-b9ea-5e36f8f5f62f None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:41.783028 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176cda27-b42d-48c4-b9ea-5e36f8f5f62f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:41.802005 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-176cda27-b42d-48c4-b9ea-5e36f8f5f62f None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:16:41.803210 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-176cda27-b42d-48c4-b9ea-5e36f8f5f62f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:16:41.871860 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 105/207] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Nov 2 07:16:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:16:41.885226 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f04f01-9c7e-42b3-ba74-32a6880ae0db None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:41.885548 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f04f01-9c7e-42b3-ba74-32a6880ae0db None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:41.885784 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f04f01-9c7e-42b3-ba74-32a6880ae0db None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:41.887387 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 103/208] 10.222.0.18 () {54 vars in 927 bytes} [Wed Nov 2 07:16:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:16:41.893254 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5aca80-366f-47c3-a3ca-d7c737865f8e None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:41.893552 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5aca80-366f-47c3-a3ca-d7c737865f8e None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:41.893806 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5aca80-366f-47c3-a3ca-d7c737865f8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:41.913685 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-ae5aca80-366f-47c3-a3ca-d7c737865f8e None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:16:41.915021 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-ae5aca80-366f-47c3-a3ca-d7c737865f8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:16:41.982124 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 106/209] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Nov 2 07:16:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:16:42.007576 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf31534-dcf4-4a47-97fd-a04d7450ef0a None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:42.007854 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf31534-dcf4-4a47-97fd-a04d7450ef0a None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:42.008122 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf31534-dcf4-4a47-97fd-a04d7450ef0a None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:42.009592 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 104/210] 10.222.0.18 () {54 vars in 927 bytes} [Wed Nov 2 07:16:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:16:42.014470 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fb1b48-569d-495d-8a2d-aa294defc7ee None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:42.014760 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fb1b48-569d-495d-8a2d-aa294defc7ee None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:42.015034 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fb1b48-569d-495d-8a2d-aa294defc7ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:42.033233 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-a4fb1b48-569d-495d-8a2d-aa294defc7ee None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:16:42.034407 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-a4fb1b48-569d-495d-8a2d-aa294defc7ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:16:42.092436 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:42.106463 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 107/211] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Nov 2 07:16:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:16:42.140537 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf31534-dcf4-4a47-97fd-a04d7450ef0a None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:42.152329 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:42.241784 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf31534-dcf4-4a47-97fd-a04d7450ef0a None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:42.252628 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fb1b48-569d-495d-8a2d-aa294defc7ee None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:42.279221 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffa5713-ae36-4f34-81a6-d9b8e40905b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:42.279848 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffa5713-ae36-4f34-81a6-d9b8e40905b6 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:42.280155 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffa5713-ae36-4f34-81a6-d9b8e40905b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:42.280409 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffa5713-ae36-4f34-81a6-d9b8e40905b6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:42.290754 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffa5713-ae36-4f34-81a6-d9b8e40905b6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:16:42.295994 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-cffa5713-ae36-4f34-81a6-d9b8e40905b6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Nov 02 07:16:42.295994 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:42.295994 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:16:42.295994 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:16:42.295994 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:16:42.295994 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:16:42.295994 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:16:42.295994 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:16:42.295994 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:16:42.295994 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:16:42.295994 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:16:42.295994 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:16:42.295994 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:16:42.295994 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:16:42.295994 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:16:42.295994 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:16:42.296944 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:16:42.296944 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:16:42.296944 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:16:42.296944 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:16:42.296944 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:16:42.296944 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:16:42.296944 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:16:42.296944 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:16:42.296944 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:16:42.296944 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:16:42.296944 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:16:42.296944 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Nov 02 07:16:42.296944 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:16:42.296944 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:16:42.296944 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:16:42.296944 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:16:42.297712 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:16:42.297712 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:16:42.297712 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:16:42.297712 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:16:42.297712 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:16:42.297712 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:16:42.297712 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:16:42.297712 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:16:42.297712 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:16:42.297712 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:16:42.297712 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Nov 02 07:16:42.297712 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Nov 02 07:16:42.297712 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 105/212] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Nov 2 07:16:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:16:42.304654 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffa5713-ae36-4f34-81a6-d9b8e40905b6 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:42.361697 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fb1b48-569d-495d-8a2d-aa294defc7ee None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:42.376412 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:42.396028 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be49a602-de0e-4351-9c80-42416f407028 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:42.396659 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be49a602-de0e-4351-9c80-42416f407028 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:42.396935 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be49a602-de0e-4351-9c80-42416f407028 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:42.397211 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be49a602-de0e-4351-9c80-42416f407028 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:42.398926 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be49a602-de0e-4351-9c80-42416f407028 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:42.400349 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be49a602-de0e-4351-9c80-42416f407028 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:42.402061 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 108/213] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Nov 2 07:16:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:42.402759 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be49a602-de0e-4351-9c80-42416f407028 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:42.410789 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffa5713-ae36-4f34-81a6-d9b8e40905b6 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:42.443316 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bb14bd-4d56-4bfa-bccb-3bebfc5948d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:42.443927 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bb14bd-4d56-4bfa-bccb-3bebfc5948d7 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:42.444748 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bb14bd-4d56-4bfa-bccb-3bebfc5948d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:42.445222 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bb14bd-4d56-4bfa-bccb-3bebfc5948d7 None admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:42.445883 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bb14bd-4d56-4bfa-bccb-3bebfc5948d7 None admin] RBAC: Adding query filter params (name=member) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:42.446183 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bb14bd-4d56-4bfa-bccb-3bebfc5948d7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:42.447632 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bb14bd-4d56-4bfa-bccb-3bebfc5948d7 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:42.455539 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 106/214] 10.222.0.18 () {56 vars in 1101 bytes} [Wed Nov 2 07:16:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:16:42.456142 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bb14bd-4d56-4bfa-bccb-3bebfc5948d7 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:42.463644 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:42.504261 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be49a602-de0e-4351-9c80-42416f407028 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:42.537411 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d70e48d-33c9-4b89-9954-0b5905e74acf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:42.538566 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d70e48d-33c9-4b89-9954-0b5905e74acf None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:42.538838 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d70e48d-33c9-4b89-9954-0b5905e74acf None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:42.539105 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d70e48d-33c9-4b89-9954-0b5905e74acf None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:42.549015 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d70e48d-33c9-4b89-9954-0b5905e74acf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:16:42.552982 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bb14bd-4d56-4bfa-bccb-3bebfc5948d7 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:42.553980 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-5d70e48d-33c9-4b89-9954-0b5905e74acf None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Nov 02 07:16:42.553980 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:42.553980 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:16:42.553980 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:16:42.553980 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:16:42.553980 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:16:42.553980 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:16:42.553980 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:16:42.553980 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:16:42.553980 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:16:42.553980 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:16:42.553980 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:16:42.553980 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:16:42.553980 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:16:42.553980 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:16:42.553980 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:16:42.554969 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:16:42.554969 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:16:42.554969 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:16:42.554969 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:16:42.554969 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:16:42.554969 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:16:42.554969 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:16:42.554969 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:16:42.554969 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:16:42.554969 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:16:42.554969 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:16:42.554969 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:16:42.554969 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:16:42.554969 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:16:42.554969 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:16:42.554969 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:16:42.555958 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:16:42.555958 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:16:42.555958 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:16:42.555958 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:16:42.555958 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:16:42.555958 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:16:42.555958 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:16:42.555958 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:16:42.555958 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:16:42.555958 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:16:42.555958 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Nov 02 07:16:42.555958 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:16:42.555958 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 109/215] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Nov 2 07:16:42 2022] GET /identity/v3/roles/reader => generated 84 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:16:42.557655 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d70e48d-33c9-4b89-9954-0b5905e74acf None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:42.581579 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:42.587547 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bc80a1-1bc7-423a-9bea-fdac6b1a1a2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:42.588154 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bc80a1-1bc7-423a-9bea-fdac6b1a1a2d None admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:42.588434 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bc80a1-1bc7-423a-9bea-fdac6b1a1a2d None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:42.588707 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bc80a1-1bc7-423a-9bea-fdac6b1a1a2d None admin] PATH_INFO: `/v3/users` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:42.589348 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bc80a1-1bc7-423a-9bea-fdac6b1a1a2d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:42.590815 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bc80a1-1bc7-423a-9bea-fdac6b1a1a2d None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:42.612093 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 107/216] 10.222.0.18 () {60 vars in 1132 bytes} [Wed Nov 2 07:16:42 2022] POST /identity/v3/users => generated 293 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:16:42.613151 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bc80a1-1bc7-423a-9bea-fdac6b1a1a2d None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:42.667342 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d70e48d-33c9-4b89-9954-0b5905e74acf None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:42.689478 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:42.708440 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3a6436-cef9-44b8-a147-2ff5ebc271e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:42.709206 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3a6436-cef9-44b8-a147-2ff5ebc271e7 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:42.709579 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3a6436-cef9-44b8-a147-2ff5ebc271e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:42.709956 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3a6436-cef9-44b8-a147-2ff5ebc271e7 None admin] PATH_INFO: `/v3/users/bae472e6896748ec89235f4831a3b757` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:42.724990 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3a6436-cef9-44b8-a147-2ff5ebc271e7 None admin] RBAC: Authorizing `identity:get_user(user_id=bae472e6896748ec89235f4831a3b757)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:42.727002 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3a6436-cef9-44b8-a147-2ff5ebc271e7 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:42.729063 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 110/217] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Nov 2 07:16:42 2022] GET /identity/v3/users/bae472e6896748ec89235f4831a3b757 => generated 293 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:42.734930 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3a6436-cef9-44b8-a147-2ff5ebc271e7 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:42.736597 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bc80a1-1bc7-423a-9bea-fdac6b1a1a2d None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:42.779811 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa637ae-1a0a-4150-b369-4e451bca4259 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:42.780462 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa637ae-1a0a-4150-b369-4e451bca4259 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:42.780707 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa637ae-1a0a-4150-b369-4e451bca4259 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:42.780877 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa637ae-1a0a-4150-b369-4e451bca4259 None admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:42.781605 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa637ae-1a0a-4150-b369-4e451bca4259 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:42.781909 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa637ae-1a0a-4150-b369-4e451bca4259 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:42.783984 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa637ae-1a0a-4150-b369-4e451bca4259 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:42.795534 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 108/218] 10.222.0.18 () {56 vars in 1101 bytes} [Wed Nov 2 07:16:42 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:42.801078 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa637ae-1a0a-4150-b369-4e451bca4259 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:42.858871 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:42.861729 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3a6436-cef9-44b8-a147-2ff5ebc271e7 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:42.907826 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66810dd0-2cf7-4848-92ce-ada9b459a729 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:42.908439 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66810dd0-2cf7-4848-92ce-ada9b459a729 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:42.908714 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66810dd0-2cf7-4848-92ce-ada9b459a729 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:42.908996 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66810dd0-2cf7-4848-92ce-ada9b459a729 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:42.909703 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66810dd0-2cf7-4848-92ce-ada9b459a729 None admin] RBAC: Adding query filter params (role.id=b494fedd00f24f0b84a529e8672a156a, user.id=bae472e6896748ec89235f4831a3b757, scope.system=all) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:42.910182 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66810dd0-2cf7-4848-92ce-ada9b459a729 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:42.911459 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66810dd0-2cf7-4848-92ce-ada9b459a729 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:42.918213 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 111/219] 10.222.0.18 () {56 vars in 1308 bytes} [Wed Nov 2 07:16:42 2022] GET /identity/v3/role_assignments?user.id=bae472e6896748ec89235f4831a3b757&scope.system=all&role.id=b494fedd00f24f0b84a529e8672a156a => generated 226 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:16:42.926747 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa637ae-1a0a-4150-b369-4e451bca4259 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:42.969190 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aab76f7-32d6-47e1-a9ff-a043a18bd5ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:42.969827 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aab76f7-32d6-47e1-a9ff-a043a18bd5ee None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:42.970103 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aab76f7-32d6-47e1-a9ff-a043a18bd5ee None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:42.970425 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aab76f7-32d6-47e1-a9ff-a043a18bd5ee None admin] PATH_INFO: `/v3/users/4c414e37665448af88c06472b37e608c` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:42.971564 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aab76f7-32d6-47e1-a9ff-a043a18bd5ee None admin] RBAC: Authorizing `identity:get_user(user_id=4c414e37665448af88c06472b37e608c)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:42.973059 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aab76f7-32d6-47e1-a9ff-a043a18bd5ee None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:42.974768 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 109/220] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Nov 2 07:16:42 2022] GET /identity/v3/users/4c414e37665448af88c06472b37e608c => generated 297 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:42.981090 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66810dd0-2cf7-4848-92ce-ada9b459a729 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:43.030082 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:43.097273 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66810dd0-2cf7-4848-92ce-ada9b459a729 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:43.135560 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e107db-1786-4ec7-928d-ef07a0aa2bc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:43.136538 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e107db-1786-4ec7-928d-ef07a0aa2bc5 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:43.137009 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e107db-1786-4ec7-928d-ef07a0aa2bc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:43.137466 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e107db-1786-4ec7-928d-ef07a0aa2bc5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:43.206763 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e107db-1786-4ec7-928d-ef07a0aa2bc5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:43.208576 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e107db-1786-4ec7-928d-ef07a0aa2bc5 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:43.254763 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:43.311934 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 112/221] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Nov 2 07:16:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Nov 02 07:16:43.317424 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aab76f7-32d6-47e1-a9ff-a043a18bd5ee None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:43.428796 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aab76f7-32d6-47e1-a9ff-a043a18bd5ee None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:43.445390 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:43.465558 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fc687d-a14f-4638-8374-09ba1adfbf0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:43.466183 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fc687d-a14f-4638-8374-09ba1adfbf0b None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:43.466497 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fc687d-a14f-4638-8374-09ba1adfbf0b None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:43.466765 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fc687d-a14f-4638-8374-09ba1adfbf0b None admin] PATH_INFO: `/v3/projects/826fa0cd5d134fcab8a9411750769a8f` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:43.467874 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fc687d-a14f-4638-8374-09ba1adfbf0b None admin] RBAC: Authorizing `identity:get_project(project_id=826fa0cd5d134fcab8a9411750769a8f)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:43.469322 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fc687d-a14f-4638-8374-09ba1adfbf0b None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:43.471141 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 110/222] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Nov 2 07:16:43 2022] GET /identity/v3/projects/826fa0cd5d134fcab8a9411750769a8f => generated 302 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:43.477296 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e107db-1786-4ec7-928d-ef07a0aa2bc5 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:43.579023 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e107db-1786-4ec7-928d-ef07a0aa2bc5 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:43.616633 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f429a82-62bb-4281-a6d5-30907acb60a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:43.617762 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f429a82-62bb-4281-a6d5-30907acb60a5 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:43.618233 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f429a82-62bb-4281-a6d5-30907acb60a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:43.618777 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f429a82-62bb-4281-a6d5-30907acb60a5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:43.619695 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f429a82-62bb-4281-a6d5-30907acb60a5 None admin] RBAC: Adding query filter params (role.id=979bdb3f4d8d4d97b1729d3162372a09, scope.project.id=826fa0cd5d134fcab8a9411750769a8f, user.id=4c414e37665448af88c06472b37e608c) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:43.620229 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f429a82-62bb-4281-a6d5-30907acb60a5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:43.621832 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f429a82-62bb-4281-a6d5-30907acb60a5 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:43.628492 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 113/223] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Nov 2 07:16:43 2022] GET /identity/v3/role_assignments?user.id=4c414e37665448af88c06472b37e608c&scope.project.id=826fa0cd5d134fcab8a9411750769a8f&role.id=979bdb3f4d8d4d97b1729d3162372a09 => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:45.048302 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b43e69-497c-4f5f-ac62-f3c28e71bfac None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:45.048852 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b43e69-497c-4f5f-ac62-f3c28e71bfac None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:45.048852 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b43e69-497c-4f5f-ac62-f3c28e71bfac None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:45.050335 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 111/224] 10.222.0.18 () {54 vars in 927 bytes} [Wed Nov 2 07:16:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:16:45.056947 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d53cbe0-e231-49b3-b843-93e22048762c None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:45.057483 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d53cbe0-e231-49b3-b843-93e22048762c None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:45.057483 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d53cbe0-e231-49b3-b843-93e22048762c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:45.065775 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:45.077040 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-4d53cbe0-e231-49b3-b843-93e22048762c None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:16:45.078209 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-4d53cbe0-e231-49b3-b843-93e22048762c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:16:45.124051 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7929ef8-c549-413d-bd24-9790fe243527 None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:45.124352 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7929ef8-c549-413d-bd24-9790fe243527 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:45.124599 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7929ef8-c549-413d-bd24-9790fe243527 None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:45.125940 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 112/225] 10.222.0.18 () {54 vars in 927 bytes} [Wed Nov 2 07:16:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Nov 02 07:16:45.130976 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b4edc1-58ee-4aea-96c8-3285e769d67e None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:45.131260 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b4edc1-58ee-4aea-96c8-3285e769d67e None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:45.131587 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b4edc1-58ee-4aea-96c8-3285e769d67e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:45.150032 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-17b4edc1-58ee-4aea-96c8-3285e769d67e None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:16:45.151239 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-17b4edc1-58ee-4aea-96c8-3285e769d67e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:16:45.152391 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 114/226] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Nov 2 07:16:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:16:45.153620 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3a1dbf-22ab-437f-ac32-7993f66ce4b4 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:45.153902 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3a1dbf-22ab-437f-ac32-7993f66ce4b4 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:45.154170 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3a1dbf-22ab-437f-ac32-7993f66ce4b4 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:45.155424 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 115/227] 10.222.0.18 () {54 vars in 927 bytes} [Wed Nov 2 07:16:45 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:16:45.160362 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6ff2ba-2384-49fb-8dc4-c473ffc8894d None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:45.160640 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6ff2ba-2384-49fb-8dc4-c473ffc8894d None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:45.160909 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6ff2ba-2384-49fb-8dc4-c473ffc8894d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:45.180117 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-3a6ff2ba-2384-49fb-8dc4-c473ffc8894d None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:16:45.181286 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-3a6ff2ba-2384-49fb-8dc4-c473ffc8894d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:16:45.226410 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 113/228] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Nov 2 07:16:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Nov 02 07:16:45.249163 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 116/229] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Nov 2 07:16:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:16:45.434208 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b4edc1-58ee-4aea-96c8-3285e769d67e None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:45.495423 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6ff2ba-2384-49fb-8dc4-c473ffc8894d None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:45.538703 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b4edc1-58ee-4aea-96c8-3285e769d67e None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:45.552430 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:45.557398 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:45.574146 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa65c7e-8cb7-4045-8005-0ff6279dd53f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:45.574808 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa65c7e-8cb7-4045-8005-0ff6279dd53f None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:45.575082 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa65c7e-8cb7-4045-8005-0ff6279dd53f None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:45.575347 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa65c7e-8cb7-4045-8005-0ff6279dd53f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:45.585045 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa65c7e-8cb7-4045-8005-0ff6279dd53f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:16:45.589834 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-9aa65c7e-8cb7-4045-8005-0ff6279dd53f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Nov 02 07:16:45.589834 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:45.589834 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:16:45.589834 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:16:45.589834 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:16:45.589834 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:16:45.589834 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:16:45.589834 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:16:45.589834 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:16:45.589834 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:16:45.589834 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:16:45.589834 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:16:45.589834 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:16:45.589834 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:16:45.589834 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:16:45.589834 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:16:45.590937 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:16:45.590937 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:16:45.590937 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:16:45.590937 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:16:45.590937 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:16:45.590937 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:16:45.590937 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:16:45.590937 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:16:45.590937 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:16:45.590937 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:16:45.590937 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:16:45.590937 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Nov 02 07:16:45.590937 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:16:45.590937 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:16:45.590937 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:16:45.590937 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:16:45.595737 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:16:45.595737 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:16:45.595737 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:16:45.595737 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:16:45.595737 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:16:45.595737 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:16:45.595737 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:16:45.595737 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:16:45.595737 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:16:45.595737 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:16:45.595737 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Nov 02 07:16:45.595737 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Nov 02 07:16:45.595737 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 114/230] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Nov 2 07:16:45 2022] GET /identity/v3/roles/reader => generated 84 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:16:45.595737 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa65c7e-8cb7-4045-8005-0ff6279dd53f None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:45.602609 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6ff2ba-2384-49fb-8dc4-c473ffc8894d None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:45.635721 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd54f5d-83ee-4c46-ad7c-d403c19604c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:45.636470 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd54f5d-83ee-4c46-ad7c-d403c19604c3 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:45.636820 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd54f5d-83ee-4c46-ad7c-d403c19604c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:45.637099 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd54f5d-83ee-4c46-ad7c-d403c19604c3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:45.638216 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd54f5d-83ee-4c46-ad7c-d403c19604c3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:45.639682 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd54f5d-83ee-4c46-ad7c-d403c19604c3 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:45.641249 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 117/231] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Nov 2 07:16:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:45.642473 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd54f5d-83ee-4c46-ad7c-d403c19604c3 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:45.693121 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa65c7e-8cb7-4045-8005-0ff6279dd53f None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:45.728733 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc58b0dc-d157-44a6-b882-0443c08dc0d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:45.729359 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc58b0dc-d157-44a6-b882-0443c08dc0d2 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:45.729639 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc58b0dc-d157-44a6-b882-0443c08dc0d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:45.729914 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc58b0dc-d157-44a6-b882-0443c08dc0d2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:45.739025 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd54f5d-83ee-4c46-ad7c-d403c19604c3 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:45.739704 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc58b0dc-d157-44a6-b882-0443c08dc0d2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:16:45.745168 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-cc58b0dc-d157-44a6-b882-0443c08dc0d2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Nov 02 07:16:45.745168 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:45.745168 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:16:45.745168 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:16:45.745168 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:16:45.745168 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:16:45.745168 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:16:45.745168 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:16:45.745168 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:16:45.745168 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:16:45.745168 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:16:45.745168 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:16:45.745168 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:16:45.745168 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:16:45.745168 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:16:45.745168 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:16:45.746033 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:16:45.746033 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:16:45.746033 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:16:45.746033 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:16:45.746033 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:16:45.746033 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:16:45.746033 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:16:45.746033 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:16:45.746033 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:16:45.746033 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:16:45.746033 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:16:45.746033 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Nov 02 07:16:45.746033 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:16:45.746033 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:16:45.746033 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:16:45.746033 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:16:45.747045 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:16:45.747045 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:16:45.747045 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:16:45.747045 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:16:45.747045 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:16:45.747045 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:16:45.747045 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:16:45.747045 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:16:45.747045 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:16:45.747045 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:16:45.747045 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Nov 02 07:16:45.747045 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Nov 02 07:16:45.747045 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 115/232] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Nov 2 07:16:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:16:45.747045 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc58b0dc-d157-44a6-b882-0443c08dc0d2 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:45.774188 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:45.779901 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c90dfce-aef4-468f-a7d3-1d68bdfa6817 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:45.780512 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c90dfce-aef4-468f-a7d3-1d68bdfa6817 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:45.780791 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c90dfce-aef4-468f-a7d3-1d68bdfa6817 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:45.781061 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c90dfce-aef4-468f-a7d3-1d68bdfa6817 None admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:45.781616 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:45.781739 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c90dfce-aef4-468f-a7d3-1d68bdfa6817 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:45.782004 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c90dfce-aef4-468f-a7d3-1d68bdfa6817 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:45.783446 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c90dfce-aef4-468f-a7d3-1d68bdfa6817 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:45.792120 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 118/233] 10.222.0.18 () {56 vars in 1101 bytes} [Wed Nov 2 07:16:45 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:45.792920 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c90dfce-aef4-468f-a7d3-1d68bdfa6817 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:45.851406 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc58b0dc-d157-44a6-b882-0443c08dc0d2 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:45.884762 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf3cff6-026a-41d1-b9af-2c51dbd2e539 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:45.885360 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf3cff6-026a-41d1-b9af-2c51dbd2e539 None admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:45.885638 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf3cff6-026a-41d1-b9af-2c51dbd2e539 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:45.885912 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf3cff6-026a-41d1-b9af-2c51dbd2e539 None admin] PATH_INFO: `/v3/groups` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:45.886557 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf3cff6-026a-41d1-b9af-2c51dbd2e539 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:45.888006 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf3cff6-026a-41d1-b9af-2c51dbd2e539 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:45.891689 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c90dfce-aef4-468f-a7d3-1d68bdfa6817 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:45.905572 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 116/234] 10.222.0.18 () {60 vars in 1134 bytes} [Wed Nov 2 07:16:45 2022] POST /identity/v3/groups => generated 229 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:16:45.907240 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf3cff6-026a-41d1-b9af-2c51dbd2e539 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:45.930820 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607f884b-688b-4052-b04e-c863a0b51055 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:45.931458 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607f884b-688b-4052-b04e-c863a0b51055 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:45.931806 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607f884b-688b-4052-b04e-c863a0b51055 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:45.932085 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607f884b-688b-4052-b04e-c863a0b51055 None admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:45.933033 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607f884b-688b-4052-b04e-c863a0b51055 None admin] RBAC: Adding query filter params (name=member) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:45.933156 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607f884b-688b-4052-b04e-c863a0b51055 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:45.934608 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607f884b-688b-4052-b04e-c863a0b51055 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:45.943900 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 119/235] 10.222.0.18 () {56 vars in 1101 bytes} [Wed Nov 2 07:16:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:45.951264 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607f884b-688b-4052-b04e-c863a0b51055 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:46.014853 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:46.019279 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:46.030491 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf3cff6-026a-41d1-b9af-2c51dbd2e539 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:46.066607 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607f884b-688b-4052-b04e-c863a0b51055 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:46.069992 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbac305-030d-4e87-85eb-4f2101b265ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:46.070489 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbac305-030d-4e87-85eb-4f2101b265ac None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:46.070698 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbac305-030d-4e87-85eb-4f2101b265ac None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:46.070945 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbac305-030d-4e87-85eb-4f2101b265ac None admin] PATH_INFO: `/v3/users/4bc3bad397bc4966b20b0530d141652e` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:46.086669 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbac305-030d-4e87-85eb-4f2101b265ac None admin] RBAC: Authorizing `identity:get_user(user_id=4bc3bad397bc4966b20b0530d141652e)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:46.088140 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbac305-030d-4e87-85eb-4f2101b265ac None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:46.089821 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 117/236] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Nov 2 07:16:45 2022] GET /identity/v3/users/4bc3bad397bc4966b20b0530d141652e => generated 293 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:46.095533 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbac305-030d-4e87-85eb-4f2101b265ac None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:46.111068 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e341a36-d972-4f2d-a79c-44451513fd87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:46.111996 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e341a36-d972-4f2d-a79c-44451513fd87 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:46.111996 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e341a36-d972-4f2d-a79c-44451513fd87 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:46.112232 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e341a36-d972-4f2d-a79c-44451513fd87 None admin] PATH_INFO: `/v3/users/bae472e6896748ec89235f4831a3b757` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:46.113379 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e341a36-d972-4f2d-a79c-44451513fd87 None admin] RBAC: Authorizing `identity:get_user(user_id=bae472e6896748ec89235f4831a3b757)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:46.114908 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e341a36-d972-4f2d-a79c-44451513fd87 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:46.117321 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 120/237] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Nov 2 07:16:45 2022] GET /identity/v3/users/bae472e6896748ec89235f4831a3b757 => generated 293 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:16:46.123326 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e341a36-d972-4f2d-a79c-44451513fd87 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:46.208335 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbac305-030d-4e87-85eb-4f2101b265ac None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:46.235029 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e341a36-d972-4f2d-a79c-44451513fd87 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:46.243350 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa2a923-8817-4e95-a273-2e0fc5e0fd0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:46.244187 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa2a923-8817-4e95-a273-2e0fc5e0fd0e None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:46.244308 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa2a923-8817-4e95-a273-2e0fc5e0fd0e None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:46.244483 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa2a923-8817-4e95-a273-2e0fc5e0fd0e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:46.245204 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa2a923-8817-4e95-a273-2e0fc5e0fd0e None admin] RBAC: Adding query filter params (role.id=979bdb3f4d8d4d97b1729d3162372a09, user.id=4bc3bad397bc4966b20b0530d141652e, scope.system=all) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:46.245507 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa2a923-8817-4e95-a273-2e0fc5e0fd0e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:46.246970 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa2a923-8817-4e95-a273-2e0fc5e0fd0e None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:46.252671 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 118/238] 10.222.0.18 () {56 vars in 1308 bytes} [Wed Nov 2 07:16:46 2022] GET /identity/v3/role_assignments?user.id=4bc3bad397bc4966b20b0530d141652e&scope.system=all&role.id=979bdb3f4d8d4d97b1729d3162372a09 => generated 226 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:16:46.262515 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:46.275128 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf2306f-b71e-46a5-8f22-bc89df1ba841 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:46.275806 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf2306f-b71e-46a5-8f22-bc89df1ba841 None admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:46.276081 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf2306f-b71e-46a5-8f22-bc89df1ba841 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:46.276349 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf2306f-b71e-46a5-8f22-bc89df1ba841 None admin] PATH_INFO: `/v3/system/users/bae472e6896748ec89235f4831a3b757/roles/b494fedd00f24f0b84a529e8672a156a` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:46.277822 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf2306f-b71e-46a5-8f22-bc89df1ba841 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=bae472e6896748ec89235f4831a3b757, role_id=b494fedd00f24f0b84a529e8672a156a)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:46.279256 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf2306f-b71e-46a5-8f22-bc89df1ba841 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:46.290225 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 121/239] 10.222.0.18 () {58 vars in 1334 bytes} [Wed Nov 2 07:16:46 2022] PUT /identity/v3/system/users/bae472e6896748ec89235f4831a3b757/roles/b494fedd00f24f0b84a529e8672a156a => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:16:48.126875 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b84479-4261-4672-acd6-67f890054b23 None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:48.127392 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b84479-4261-4672-acd6-67f890054b23 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:48.127392 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b84479-4261-4672-acd6-67f890054b23 None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:48.128806 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 119/240] 10.222.0.18 () {54 vars in 927 bytes} [Wed Nov 2 07:16:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:16:48.134626 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d225c0f-65af-4f3d-bb0a-7137891236dd None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:48.135084 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d225c0f-65af-4f3d-bb0a-7137891236dd None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:48.135161 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d225c0f-65af-4f3d-bb0a-7137891236dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:48.153440 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-5d225c0f-65af-4f3d-bb0a-7137891236dd None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:16:48.154631 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-5d225c0f-65af-4f3d-bb0a-7137891236dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:16:48.224374 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 122/241] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Nov 2 07:16:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Nov 02 07:16:48.397930 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6763070-e119-4c6b-bda7-577d5368b545 None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:48.398214 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6763070-e119-4c6b-bda7-577d5368b545 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:48.398421 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6763070-e119-4c6b-bda7-577d5368b545 None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:48.399898 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 120/242] 10.222.0.18 () {54 vars in 927 bytes} [Wed Nov 2 07:16:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:16:48.404964 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7d5ecd-e3f8-4801-bdb1-36f3c4f5e756 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:48.405249 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7d5ecd-e3f8-4801-bdb1-36f3c4f5e756 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:48.405524 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7d5ecd-e3f8-4801-bdb1-36f3c4f5e756 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:48.423737 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-1d7d5ecd-e3f8-4801-bdb1-36f3c4f5e756 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:16:48.424893 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-1d7d5ecd-e3f8-4801-bdb1-36f3c4f5e756 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:16:48.462279 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5664f549-5002-45e5-8eff-25016040a6ce None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:48.462599 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5664f549-5002-45e5-8eff-25016040a6ce None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:48.462867 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5664f549-5002-45e5-8eff-25016040a6ce None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:48.464334 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 121/243] 10.222.0.18 () {54 vars in 927 bytes} [Wed Nov 2 07:16:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Nov 02 07:16:48.469725 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecab728-6a5a-44ee-8d71-1071872a8a48 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:48.470010 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecab728-6a5a-44ee-8d71-1071872a8a48 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:48.470286 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecab728-6a5a-44ee-8d71-1071872a8a48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:48.488796 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-5ecab728-6a5a-44ee-8d71-1071872a8a48 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:16:48.489991 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-5ecab728-6a5a-44ee-8d71-1071872a8a48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:16:48.492256 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 123/244] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Nov 2 07:16:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:16:48.495042 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7d5ecd-e3f8-4801-bdb1-36f3c4f5e756 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:48.501815 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:48.548443 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:48.561890 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 122/245] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Nov 2 07:16:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:16:48.603078 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7d5ecd-e3f8-4801-bdb1-36f3c4f5e756 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:48.637392 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f42b73-459c-40c8-85f9-3065a444a64d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:48.638023 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f42b73-459c-40c8-85f9-3065a444a64d None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:48.638302 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f42b73-459c-40c8-85f9-3065a444a64d None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:48.638604 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f42b73-459c-40c8-85f9-3065a444a64d None admin] PATH_INFO: `/v3/domains/default` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:48.639753 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f42b73-459c-40c8-85f9-3065a444a64d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:48.642418 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f42b73-459c-40c8-85f9-3065a444a64d None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:48.642882 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 124/246] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Nov 2 07:16:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:16:48.649207 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecab728-6a5a-44ee-8d71-1071872a8a48 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:48.749858 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecab728-6a5a-44ee-8d71-1071872a8a48 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:48.762318 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f42b73-459c-40c8-85f9-3065a444a64d None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:48.790454 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925aabc4-3b8d-4fb4-9611-09205c96b8cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:48.791363 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925aabc4-3b8d-4fb4-9611-09205c96b8cf None admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:48.791644 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925aabc4-3b8d-4fb4-9611-09205c96b8cf None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:48.791914 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925aabc4-3b8d-4fb4-9611-09205c96b8cf None admin] PATH_INFO: `/v3/groups` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:48.792725 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925aabc4-3b8d-4fb4-9611-09205c96b8cf None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:48.794820 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925aabc4-3b8d-4fb4-9611-09205c96b8cf None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:48.818495 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 123/247] 10.222.0.18 () {60 vars in 1134 bytes} [Wed Nov 2 07:16:48 2022] POST /identity/v3/groups => generated 226 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 07:16:48.836482 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925aabc4-3b8d-4fb4-9611-09205c96b8cf None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:48.848659 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:48.884218 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f42b73-459c-40c8-85f9-3065a444a64d None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:48.903685 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:48.925993 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc2f845-eb6f-463c-9013-e4ebd030fd3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:48.926664 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc2f845-eb6f-463c-9013-e4ebd030fd3c None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:48.926940 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc2f845-eb6f-463c-9013-e4ebd030fd3c None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:48.927206 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc2f845-eb6f-463c-9013-e4ebd030fd3c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:48.939015 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc2f845-eb6f-463c-9013-e4ebd030fd3c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:16:48.944076 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-ddc2f845-eb6f-463c-9013-e4ebd030fd3c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Nov 02 07:16:48.944076 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:48.944076 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:16:48.944076 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:16:48.944076 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:16:48.944076 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:16:48.944076 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:16:48.944076 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:16:48.944076 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:16:48.944076 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:16:48.944076 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:16:48.944076 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:16:48.944076 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:16:48.944076 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:16:48.944076 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:16:48.944076 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:16:48.945822 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:16:48.945822 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:16:48.945822 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:16:48.945822 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:16:48.945822 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:16:48.945822 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:16:48.945822 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:16:48.945822 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:16:48.945822 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:16:48.945822 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:16:48.945822 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:16:48.945822 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:16:48.945822 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:16:48.945822 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:16:48.945822 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:16:48.945822 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:16:48.946678 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:16:48.946678 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:16:48.946678 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:16:48.946678 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:16:48.946678 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:16:48.946678 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:16:48.946678 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:16:48.946678 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:16:48.946678 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:16:48.946678 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:16:48.946678 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Nov 02 07:16:48.946678 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:16:48.946678 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 125/248] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Nov 2 07:16:48 2022] GET /identity/v3/roles/reader => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 07:16:48.953888 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc2f845-eb6f-463c-9013-e4ebd030fd3c None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:48.961098 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925aabc4-3b8d-4fb4-9611-09205c96b8cf None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:48.999097 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d56f98-75d1-4519-ad81-b3e875a4e2c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:48.999918 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d56f98-75d1-4519-ad81-b3e875a4e2c5 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:49.000015 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d56f98-75d1-4519-ad81-b3e875a4e2c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:49.000281 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d56f98-75d1-4519-ad81-b3e875a4e2c5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:49.013496 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d56f98-75d1-4519-ad81-b3e875a4e2c5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:16:49.020283 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-68d56f98-75d1-4519-ad81-b3e875a4e2c5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Nov 02 07:16:49.020283 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:49.020283 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:16:49.020283 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:16:49.020283 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:16:49.020283 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:16:49.020283 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:16:49.020283 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:16:49.020283 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:16:49.020283 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:16:49.020283 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:16:49.020283 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:16:49.020283 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:16:49.020283 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:16:49.020283 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:16:49.020283 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:16:49.021153 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:16:49.021153 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:16:49.021153 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:16:49.021153 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:16:49.021153 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:16:49.021153 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:16:49.021153 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:16:49.021153 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:16:49.021153 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:16:49.021153 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:16:49.021153 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:16:49.021153 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Nov 02 07:16:49.021153 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:16:49.021153 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:16:49.021153 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:16:49.021153 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:16:49.022941 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:16:49.022941 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:16:49.022941 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:16:49.022941 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:16:49.022941 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:16:49.022941 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:16:49.022941 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:16:49.022941 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:16:49.022941 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:16:49.022941 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:16:49.022941 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Nov 02 07:16:49.022941 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Nov 02 07:16:49.022941 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 124/249] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Nov 2 07:16:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 07:16:49.027773 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d56f98-75d1-4519-ad81-b3e875a4e2c5 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:49.069465 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc2f845-eb6f-463c-9013-e4ebd030fd3c None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:49.117166 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08cce16-5a64-4c5c-8ea2-e3e01557260b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:49.118065 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08cce16-5a64-4c5c-8ea2-e3e01557260b None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:49.118551 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08cce16-5a64-4c5c-8ea2-e3e01557260b None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:49.118979 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08cce16-5a64-4c5c-8ea2-e3e01557260b None admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:49.119982 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08cce16-5a64-4c5c-8ea2-e3e01557260b None admin] RBAC: Adding query filter params (name=reader) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:49.120206 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08cce16-5a64-4c5c-8ea2-e3e01557260b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:49.121131 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:49.122064 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08cce16-5a64-4c5c-8ea2-e3e01557260b None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:49.132733 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 126/250] 10.222.0.18 () {56 vars in 1101 bytes} [Wed Nov 2 07:16:48 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:49.138147 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08cce16-5a64-4c5c-8ea2-e3e01557260b None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:49.165219 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d56f98-75d1-4519-ad81-b3e875a4e2c5 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:49.183634 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:49.213142 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db1d235-6373-4e0e-a9f6-f68c181770cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:49.213992 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db1d235-6373-4e0e-a9f6-f68c181770cf None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:49.214469 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db1d235-6373-4e0e-a9f6-f68c181770cf None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:49.214877 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db1d235-6373-4e0e-a9f6-f68c181770cf None admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:49.215742 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db1d235-6373-4e0e-a9f6-f68c181770cf None admin] RBAC: Adding query filter params (name=member) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:49.216213 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db1d235-6373-4e0e-a9f6-f68c181770cf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:49.217780 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db1d235-6373-4e0e-a9f6-f68c181770cf None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:49.230721 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 125/251] 10.222.0.18 () {56 vars in 1101 bytes} [Wed Nov 2 07:16:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:16:49.236467 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db1d235-6373-4e0e-a9f6-f68c181770cf None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:49.266851 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08cce16-5a64-4c5c-8ea2-e3e01557260b None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:49.320048 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945d3bf6-518e-4e20-bdd7-548e29b216fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:49.321108 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945d3bf6-518e-4e20-bdd7-548e29b216fe None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:49.321502 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945d3bf6-518e-4e20-bdd7-548e29b216fe None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:49.321960 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945d3bf6-518e-4e20-bdd7-548e29b216fe None admin] PATH_INFO: `/v3/users/4bc3bad397bc4966b20b0530d141652e` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:49.323963 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945d3bf6-518e-4e20-bdd7-548e29b216fe None admin] RBAC: Authorizing `identity:get_user(user_id=4bc3bad397bc4966b20b0530d141652e)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:49.326000 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945d3bf6-518e-4e20-bdd7-548e29b216fe None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:49.328365 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 127/252] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Nov 2 07:16:49 2022] GET /identity/v3/users/4bc3bad397bc4966b20b0530d141652e => generated 293 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:16:49.334132 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945d3bf6-518e-4e20-bdd7-548e29b216fe None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:49.369843 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db1d235-6373-4e0e-a9f6-f68c181770cf None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:49.400021 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:49.411136 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf6ed74-33dc-434f-a794-7432cd67b19e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:49.412107 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf6ed74-33dc-434f-a794-7432cd67b19e None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:49.413138 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf6ed74-33dc-434f-a794-7432cd67b19e None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:49.413620 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf6ed74-33dc-434f-a794-7432cd67b19e None admin] PATH_INFO: `/v3/users/bae472e6896748ec89235f4831a3b757` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:49.416892 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf6ed74-33dc-434f-a794-7432cd67b19e None admin] RBAC: Authorizing `identity:get_user(user_id=bae472e6896748ec89235f4831a3b757)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:49.418546 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf6ed74-33dc-434f-a794-7432cd67b19e None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:49.420316 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 126/253] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Nov 2 07:16:49 2022] GET /identity/v3/users/bae472e6896748ec89235f4831a3b757 => generated 293 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:16:49.426387 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf6ed74-33dc-434f-a794-7432cd67b19e None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:49.454033 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945d3bf6-518e-4e20-bdd7-548e29b216fe None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:49.463973 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:49.503337 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dea41e-7ec4-4af7-95d5-d983dff782c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:49.504282 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dea41e-7ec4-4af7-95d5-d983dff782c3 None admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:49.504567 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dea41e-7ec4-4af7-95d5-d983dff782c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:49.504840 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dea41e-7ec4-4af7-95d5-d983dff782c3 None admin] PATH_INFO: `/v3/system/users/4bc3bad397bc4966b20b0530d141652e/roles/979bdb3f4d8d4d97b1729d3162372a09` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:49.507360 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0dea41e-7ec4-4af7-95d5-d983dff782c3 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=4bc3bad397bc4966b20b0530d141652e, role_id=979bdb3f4d8d4d97b1729d3162372a09)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:49.508744 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0dea41e-7ec4-4af7-95d5-d983dff782c3 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:49.518734 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 128/254] 10.222.0.18 () {58 vars in 1334 bytes} [Wed Nov 2 07:16:49 2022] PUT /identity/v3/system/users/4bc3bad397bc4966b20b0530d141652e/roles/979bdb3f4d8d4d97b1729d3162372a09 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 07:16:49.552371 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf6ed74-33dc-434f-a794-7432cd67b19e None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:49.599890 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eaaa23f-a3aa-4dba-87bd-c3ed9d3c0f5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:49.600912 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaaa23f-a3aa-4dba-87bd-c3ed9d3c0f5d None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:49.601330 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaaa23f-a3aa-4dba-87bd-c3ed9d3c0f5d None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:49.601792 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaaa23f-a3aa-4dba-87bd-c3ed9d3c0f5d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:49.602757 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eaaa23f-a3aa-4dba-87bd-c3ed9d3c0f5d None admin] RBAC: Adding query filter params (role.id=b494fedd00f24f0b84a529e8672a156a, user.id=bae472e6896748ec89235f4831a3b757, scope.system=all) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:49.603478 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eaaa23f-a3aa-4dba-87bd-c3ed9d3c0f5d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:49.605381 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eaaa23f-a3aa-4dba-87bd-c3ed9d3c0f5d None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:49.624921 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 127/255] 10.222.0.18 () {56 vars in 1308 bytes} [Wed Nov 2 07:16:49 2022] GET /identity/v3/role_assignments?user.id=bae472e6896748ec89235f4831a3b757&scope.system=all&role.id=b494fedd00f24f0b84a529e8672a156a => generated 510 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:16:51.325243 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf05600-cdc5-4bea-85a4-a77eb2fb57c5 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:51.325762 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf05600-cdc5-4bea-85a4-a77eb2fb57c5 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:51.325762 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf05600-cdc5-4bea-85a4-a77eb2fb57c5 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:51.327416 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 129/256] 10.222.0.18 () {54 vars in 927 bytes} [Wed Nov 2 07:16:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:16:51.335156 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcb8085-f5de-4e12-b64e-7d26756d1e3a None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:51.335629 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcb8085-f5de-4e12-b64e-7d26756d1e3a None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:51.335703 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcb8085-f5de-4e12-b64e-7d26756d1e3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:51.359833 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-dfcb8085-f5de-4e12-b64e-7d26756d1e3a None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:16:51.361024 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-dfcb8085-f5de-4e12-b64e-7d26756d1e3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:16:51.432284 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:51.447869 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 128/257] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Nov 2 07:16:51 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Nov 02 07:16:51.558129 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99f1ec5-5996-41c9-80a7-b522eaa10fff None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:51.558823 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99f1ec5-5996-41c9-80a7-b522eaa10fff None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:51.559293 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99f1ec5-5996-41c9-80a7-b522eaa10fff None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:51.561084 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 130/258] 10.222.0.18 () {54 vars in 927 bytes} [Wed Nov 2 07:16:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:16:51.565987 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48dfdfc-5268-4ba8-8aa2-b5fddd5693ef None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:51.566272 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48dfdfc-5268-4ba8-8aa2-b5fddd5693ef None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:51.566570 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48dfdfc-5268-4ba8-8aa2-b5fddd5693ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:51.591095 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-f48dfdfc-5268-4ba8-8aa2-b5fddd5693ef None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:16:51.592253 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-f48dfdfc-5268-4ba8-8aa2-b5fddd5693ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:16:51.605663 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 129/259] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Nov 2 07:16:51 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Nov 02 07:16:51.935944 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99f1ec5-5996-41c9-80a7-b522eaa10fff None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:51.949277 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99f1ec5-5996-41c9-80a7-b522eaa10fff None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:51.958454 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef876be3-fe41-4567-b53a-62e7b4289b0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:51.959815 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef876be3-fe41-4567-b53a-62e7b4289b0b admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:51.960240 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef876be3-fe41-4567-b53a-62e7b4289b0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:51.960650 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef876be3-fe41-4567-b53a-62e7b4289b0b admin admin] PATH_INFO: `/v3/roles/member` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:51.977851 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef876be3-fe41-4567-b53a-62e7b4289b0b admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:16:51.983197 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-ef876be3-fe41-4567-b53a-62e7b4289b0b admin admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Nov 02 07:16:51.983197 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:51.983197 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:16:51.983197 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:16:51.983197 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:16:51.983197 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:16:51.983197 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:16:51.983197 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:16:51.983197 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:16:51.983197 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:16:51.983197 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:16:51.983197 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:16:51.983197 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:16:51.983197 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:16:51.983197 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:16:51.983197 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:16:51.984272 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:16:51.984272 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:16:51.984272 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:16:51.984272 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:16:51.984272 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:16:51.984272 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:16:51.984272 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:16:51.984272 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:16:51.984272 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:16:51.984272 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:16:51.984272 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:16:51.984272 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:16:51.984272 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:16:51.984272 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:16:51.984272 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:16:51.984272 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:16:51.985243 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:16:51.985243 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:16:51.985243 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:16:51.985243 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:16:51.985243 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:16:51.985243 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:16:51.985243 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:16:51.985243 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:16:51.985243 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:16:51.985243 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:16:51.985243 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Nov 02 07:16:51.985243 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:16:51.987472 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 131/260] 10.222.0.18 () {56 vars in 1120 bytes} [Wed Nov 2 07:16:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 52 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:16:51.993175 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48dfdfc-5268-4ba8-8aa2-b5fddd5693ef None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:52.008383 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48dfdfc-5268-4ba8-8aa2-b5fddd5693ef None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:52.023721 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659e4744-0e30-4470-89f5-3782af945fa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:52.024457 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659e4744-0e30-4470-89f5-3782af945fa0 admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:52.024811 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659e4744-0e30-4470-89f5-3782af945fa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:52.025157 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659e4744-0e30-4470-89f5-3782af945fa0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:52.025906 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659e4744-0e30-4470-89f5-3782af945fa0 admin admin] RBAC: Adding query filter params (name=member) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:52.026297 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659e4744-0e30-4470-89f5-3782af945fa0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:52.028220 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: WARNING py.warnings [None req-659e4744-0e30-4470-89f5-3782af945fa0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Nov 02 07:16:52.028220 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: warnings.warn(msg) Nov 02 07:16:52.028220 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: Nov 02 07:16:52.029221 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659e4744-0e30-4470-89f5-3782af945fa0 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:52.040844 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 130/261] 10.222.0.18 () {56 vars in 1122 bytes} [Wed Nov 2 07:16:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:52.046344 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef876be3-fe41-4567-b53a-62e7b4289b0b admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:52.061811 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef876be3-fe41-4567-b53a-62e7b4289b0b admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:52.071532 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15ea1e9-3a24-4a12-a779-cf307bc1cef6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:52.072455 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15ea1e9-3a24-4a12-a779-cf307bc1cef6 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:52.072911 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15ea1e9-3a24-4a12-a779-cf307bc1cef6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:52.073354 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15ea1e9-3a24-4a12-a779-cf307bc1cef6 admin admin] PATH_INFO: `/v3/groups/735ae239bcf0421c89bf6c19a919d4ba` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:52.087855 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15ea1e9-3a24-4a12-a779-cf307bc1cef6 admin admin] RBAC: Authorizing `identity:get_group(group_id=735ae239bcf0421c89bf6c19a919d4ba)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:52.089911 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: WARNING py.warnings [None req-b15ea1e9-3a24-4a12-a779-cf307bc1cef6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_group": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.group.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Nov 02 07:16:52.089911 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: warnings.warn(msg) Nov 02 07:16:52.089911 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: Nov 02 07:16:52.091524 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15ea1e9-3a24-4a12-a779-cf307bc1cef6 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:52.093694 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 132/262] 10.222.0.18 () {56 vars in 1201 bytes} [Wed Nov 2 07:16:52 2022] GET /identity/v3/groups/735ae239bcf0421c89bf6c19a919d4ba => generated 226 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:52.099053 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659e4744-0e30-4470-89f5-3782af945fa0 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:52.115051 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659e4744-0e30-4470-89f5-3782af945fa0 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:52.123711 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5af4f30-8997-41ad-a848-2dffd2d6fa6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:52.124505 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5af4f30-8997-41ad-a848-2dffd2d6fa6b admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:52.124858 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5af4f30-8997-41ad-a848-2dffd2d6fa6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:52.125206 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5af4f30-8997-41ad-a848-2dffd2d6fa6b admin admin] PATH_INFO: `/v3/projects/a6b063398180486c85fc258b5a513046` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:52.126466 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5af4f30-8997-41ad-a848-2dffd2d6fa6b admin admin] RBAC: Authorizing `identity:get_project(project_id=a6b063398180486c85fc258b5a513046)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:52.128626 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5af4f30-8997-41ad-a848-2dffd2d6fa6b admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:52.130041 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15ea1e9-3a24-4a12-a779-cf307bc1cef6 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:52.130707 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 131/263] 10.222.0.18 () {56 vars in 1207 bytes} [Wed Nov 2 07:16:52 2022] GET /identity/v3/projects/a6b063398180486c85fc258b5a513046 => generated 298 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:52.136388 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5af4f30-8997-41ad-a848-2dffd2d6fa6b admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:52.153355 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5af4f30-8997-41ad-a848-2dffd2d6fa6b admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:52.155982 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15ea1e9-3a24-4a12-a779-cf307bc1cef6 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:52.167122 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20317cb7-88c7-4dfb-a7fe-c37acbfc39d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:52.167817 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20317cb7-88c7-4dfb-a7fe-c37acbfc39d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:52.168096 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20317cb7-88c7-4dfb-a7fe-c37acbfc39d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:52.168367 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20317cb7-88c7-4dfb-a7fe-c37acbfc39d1 admin admin] PATH_INFO: `/v3/projects/a6b063398180486c85fc258b5a513046/groups/735ae239bcf0421c89bf6c19a919d4ba/roles/b494fedd00f24f0b84a529e8672a156a` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:52.170570 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed09b1eb-8769-4c21-9b2e-9bde39fe40ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:52.171206 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed09b1eb-8769-4c21-9b2e-9bde39fe40ea admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:52.171485 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed09b1eb-8769-4c21-9b2e-9bde39fe40ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:52.171753 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed09b1eb-8769-4c21-9b2e-9bde39fe40ea admin admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:52.172881 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20317cb7-88c7-4dfb-a7fe-c37acbfc39d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6b063398180486c85fc258b5a513046, group_id=735ae239bcf0421c89bf6c19a919d4ba, role_id=b494fedd00f24f0b84a529e8672a156a)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:52.173913 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: WARNING py.warnings [None req-20317cb7-88c7-4dfb-a7fe-c37acbfc39d1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Nov 02 07:16:52.173913 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: warnings.warn(msg) Nov 02 07:16:52.173913 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: Nov 02 07:16:52.175900 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20317cb7-88c7-4dfb-a7fe-c37acbfc39d1 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:52.195183 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed09b1eb-8769-4c21-9b2e-9bde39fe40ea admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:16:52.207524 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 132/264] 10.222.0.18 () {58 vars in 1463 bytes} [Wed Nov 2 07:16:52 2022] PUT /identity/v3/projects/a6b063398180486c85fc258b5a513046/groups/735ae239bcf0421c89bf6c19a919d4ba/roles/b494fedd00f24f0b84a529e8672a156a => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 07:16:52.207679 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-ed09b1eb-8769-4c21-9b2e-9bde39fe40ea admin admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Nov 02 07:16:52.207679 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:52.207679 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:16:52.207679 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:16:52.207679 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:16:52.207679 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:16:52.207679 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:16:52.207679 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:16:52.207679 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:16:52.207679 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:16:52.207679 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:16:52.207679 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:16:52.207679 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:16:52.207679 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:16:52.207679 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:16:52.207679 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:16:52.208505 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:16:52.208505 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:16:52.208505 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:16:52.208505 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:16:52.208505 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:16:52.208505 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:16:52.208505 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:16:52.208505 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:16:52.208505 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:16:52.208505 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:16:52.208505 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:16:52.208505 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:16:52.208505 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:16:52.208505 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:16:52.208505 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:16:52.208505 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:16:52.209313 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:16:52.209313 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:16:52.209313 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:16:52.209313 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:16:52.209313 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:16:52.209313 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:16:52.209313 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:16:52.209313 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:16:52.209313 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:16:52.209313 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:16:52.209313 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Nov 02 07:16:52.209313 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:16:52.209313 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 133/265] 10.222.0.18 () {56 vars in 1135 bytes} [Wed Nov 2 07:16:52 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 79 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 07:16:52.215109 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20317cb7-88c7-4dfb-a7fe-c37acbfc39d1 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:52.295046 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20317cb7-88c7-4dfb-a7fe-c37acbfc39d1 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:52.299879 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d19901-9a12-4949-a403-2fd6e2e99d64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:52.300555 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d19901-9a12-4949-a403-2fd6e2e99d64 admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:52.300891 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d19901-9a12-4949-a403-2fd6e2e99d64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:52.301167 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d19901-9a12-4949-a403-2fd6e2e99d64 admin admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:52.301905 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d19901-9a12-4949-a403-2fd6e2e99d64 admin admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:52.302263 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d19901-9a12-4949-a403-2fd6e2e99d64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:52.304952 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d19901-9a12-4949-a403-2fd6e2e99d64 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:52.314753 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 133/266] 10.222.0.18 () {56 vars in 1132 bytes} [Wed Nov 2 07:16:52 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:52.321305 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed09b1eb-8769-4c21-9b2e-9bde39fe40ea admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:52.333446 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed09b1eb-8769-4c21-9b2e-9bde39fe40ea admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:52.339859 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cca8f0-58c5-4970-9241-5ce296d502fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:52.340622 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cca8f0-58c5-4970-9241-5ce296d502fe admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:52.340909 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cca8f0-58c5-4970-9241-5ce296d502fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:52.341181 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cca8f0-58c5-4970-9241-5ce296d502fe admin admin] PATH_INFO: `/v3/groups/735ae239bcf0421c89bf6c19a919d4ba` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:52.342680 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cca8f0-58c5-4970-9241-5ce296d502fe admin admin] RBAC: Authorizing `identity:get_group(group_id=735ae239bcf0421c89bf6c19a919d4ba)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:52.344847 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cca8f0-58c5-4970-9241-5ce296d502fe admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:52.347176 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 134/267] 10.222.0.18 () {56 vars in 1201 bytes} [Wed Nov 2 07:16:52 2022] GET /identity/v3/groups/735ae239bcf0421c89bf6c19a919d4ba => generated 226 bytes in 27 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:52.354831 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d19901-9a12-4949-a403-2fd6e2e99d64 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:52.366647 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d19901-9a12-4949-a403-2fd6e2e99d64 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:52.371652 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177270ff-d99a-45a0-b6f3-47e934403435 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:52.372343 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177270ff-d99a-45a0-b6f3-47e934403435 admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:52.372626 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177270ff-d99a-45a0-b6f3-47e934403435 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:52.372952 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177270ff-d99a-45a0-b6f3-47e934403435 admin admin] PATH_INFO: `/v3/projects/a6b063398180486c85fc258b5a513046` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:52.374179 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177270ff-d99a-45a0-b6f3-47e934403435 admin admin] RBAC: Authorizing `identity:get_project(project_id=a6b063398180486c85fc258b5a513046)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:52.376525 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177270ff-d99a-45a0-b6f3-47e934403435 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:52.378705 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 134/268] 10.222.0.18 () {56 vars in 1207 bytes} [Wed Nov 2 07:16:52 2022] GET /identity/v3/projects/a6b063398180486c85fc258b5a513046 => generated 298 bytes in 24 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:52.384617 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cca8f0-58c5-4970-9241-5ce296d502fe admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:52.396426 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cca8f0-58c5-4970-9241-5ce296d502fe admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:52.402805 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cfbfd6-e767-45d4-b1a4-5844b74c1d89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:52.403579 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cfbfd6-e767-45d4-b1a4-5844b74c1d89 admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:52.403857 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cfbfd6-e767-45d4-b1a4-5844b74c1d89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:52.404128 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cfbfd6-e767-45d4-b1a4-5844b74c1d89 admin admin] PATH_INFO: `/v3/projects/a6b063398180486c85fc258b5a513046/groups/735ae239bcf0421c89bf6c19a919d4ba/roles/8cf467a4f06c44e0b77499aeababdcc3` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:52.407861 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cfbfd6-e767-45d4-b1a4-5844b74c1d89 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6b063398180486c85fc258b5a513046, group_id=735ae239bcf0421c89bf6c19a919d4ba, role_id=8cf467a4f06c44e0b77499aeababdcc3)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:52.408971 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: WARNING py.warnings [None req-f8cfbfd6-e767-45d4-b1a4-5844b74c1d89 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Nov 02 07:16:52.408971 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: warnings.warn(msg) Nov 02 07:16:52.408971 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: Nov 02 07:16:52.411263 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cfbfd6-e767-45d4-b1a4-5844b74c1d89 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:52.443639 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 135/269] 10.222.0.18 () {58 vars in 1463 bytes} [Wed Nov 2 07:16:52 2022] PUT /identity/v3/projects/a6b063398180486c85fc258b5a513046/groups/735ae239bcf0421c89bf6c19a919d4ba/roles/8cf467a4f06c44e0b77499aeababdcc3 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:16:52.773189 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fbfc87-bd3e-4c4b-b1e8-f1815e7408d3 None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:52.773582 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fbfc87-bd3e-4c4b-b1e8-f1815e7408d3 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:52.773936 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fbfc87-bd3e-4c4b-b1e8-f1815e7408d3 None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:52.775710 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 135/270] 10.222.0.18 () {54 vars in 927 bytes} [Wed Nov 2 07:16:52 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:16:52.777307 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa31f9a-e179-4ceb-8526-c9d1b0edf67d None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:52.777654 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa31f9a-e179-4ceb-8526-c9d1b0edf67d None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:52.777936 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa31f9a-e179-4ceb-8526-c9d1b0edf67d None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:52.779432 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 136/271] 10.222.0.18 () {54 vars in 927 bytes} [Wed Nov 2 07:16:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:16:52.782155 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330ffdb1-865c-40d3-a2be-b46697d335cd None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:52.782466 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330ffdb1-865c-40d3-a2be-b46697d335cd None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:52.782741 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330ffdb1-865c-40d3-a2be-b46697d335cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:52.786954 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1974f8-fd41-4e47-afbc-c1707858175a None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:52.787233 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1974f8-fd41-4e47-afbc-c1707858175a None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:52.788139 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1974f8-fd41-4e47-afbc-c1707858175a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:52.814949 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-330ffdb1-865c-40d3-a2be-b46697d335cd None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:16:52.817239 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-330ffdb1-865c-40d3-a2be-b46697d335cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:16:52.830593 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-fc1974f8-fd41-4e47-afbc-c1707858175a None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:16:52.832097 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-fc1974f8-fd41-4e47-afbc-c1707858175a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:16:52.902135 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 136/272] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Nov 2 07:16:52 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Nov 02 07:16:52.944251 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 137/273] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Nov 2 07:16:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:16:53.400710 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330ffdb1-865c-40d3-a2be-b46697d335cd None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:53.414445 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330ffdb1-865c-40d3-a2be-b46697d335cd None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:53.419440 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157595fc-6d12-4631-92c2-0e31d88dfa40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:53.420134 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157595fc-6d12-4631-92c2-0e31d88dfa40 admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:53.420416 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157595fc-6d12-4631-92c2-0e31d88dfa40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:53.420686 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157595fc-6d12-4631-92c2-0e31d88dfa40 admin admin] PATH_INFO: `/v3/roles/member` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:53.436284 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157595fc-6d12-4631-92c2-0e31d88dfa40 admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:16:53.441195 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-157595fc-6d12-4631-92c2-0e31d88dfa40 admin admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Nov 02 07:16:53.441195 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:53.441195 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:16:53.441195 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:16:53.441195 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:16:53.441195 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:16:53.441195 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:16:53.441195 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:16:53.441195 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:16:53.441195 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:16:53.441195 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:16:53.441195 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:16:53.441195 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:16:53.441195 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:16:53.441195 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:16:53.441195 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:16:53.442073 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:16:53.442073 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:16:53.442073 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:16:53.442073 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:16:53.442073 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:16:53.442073 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:16:53.442073 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:16:53.442073 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:16:53.442073 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:16:53.442073 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:16:53.442073 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:16:53.442073 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Nov 02 07:16:53.442073 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:16:53.442073 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:16:53.442073 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:16:53.442073 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:16:53.442900 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:16:53.442900 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:16:53.442900 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:16:53.442900 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:16:53.442900 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:16:53.442900 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:16:53.442900 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:16:53.442900 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:16:53.442900 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:16:53.442900 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:16:53.442900 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Nov 02 07:16:53.442900 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Nov 02 07:16:53.442900 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 137/274] 10.222.0.18 () {56 vars in 1120 bytes} [Wed Nov 2 07:16:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 42 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:16:53.448288 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1974f8-fd41-4e47-afbc-c1707858175a None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:53.464473 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1974f8-fd41-4e47-afbc-c1707858175a None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:53.476335 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7676f486-6e5e-46c4-a562-1583b4894303 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:53.477181 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7676f486-6e5e-46c4-a562-1583b4894303 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:53.477650 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7676f486-6e5e-46c4-a562-1583b4894303 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:53.478414 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7676f486-6e5e-46c4-a562-1583b4894303 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:53.479269 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7676f486-6e5e-46c4-a562-1583b4894303 admin admin] RBAC: Adding query filter params (name=member) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:53.479757 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7676f486-6e5e-46c4-a562-1583b4894303 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:53.480943 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: WARNING py.warnings [None req-7676f486-6e5e-46c4-a562-1583b4894303 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Nov 02 07:16:53.480943 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: warnings.warn(msg) Nov 02 07:16:53.480943 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: Nov 02 07:16:53.482201 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7676f486-6e5e-46c4-a562-1583b4894303 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:53.498013 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157595fc-6d12-4631-92c2-0e31d88dfa40 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:53.500519 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 138/275] 10.222.0.18 () {56 vars in 1122 bytes} [Wed Nov 2 07:16:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:53.507508 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7676f486-6e5e-46c4-a562-1583b4894303 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:53.524049 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7676f486-6e5e-46c4-a562-1583b4894303 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:53.535576 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff46fdb-7cef-4584-960d-ab5143e6aa20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:53.536552 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff46fdb-7cef-4584-960d-ab5143e6aa20 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:53.537027 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff46fdb-7cef-4584-960d-ab5143e6aa20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:53.537621 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff46fdb-7cef-4584-960d-ab5143e6aa20 admin admin] PATH_INFO: `/v3/groups/735ae239bcf0421c89bf6c19a919d4ba` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:53.543081 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff46fdb-7cef-4584-960d-ab5143e6aa20 admin admin] RBAC: Authorizing `identity:get_group(group_id=735ae239bcf0421c89bf6c19a919d4ba)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:53.545273 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff46fdb-7cef-4584-960d-ab5143e6aa20 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:53.547169 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 139/276] 10.222.0.18 () {56 vars in 1201 bytes} [Wed Nov 2 07:16:53 2022] GET /identity/v3/groups/735ae239bcf0421c89bf6c19a919d4ba => generated 226 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:16:53.554444 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff46fdb-7cef-4584-960d-ab5143e6aa20 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:53.558773 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:53.571809 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff46fdb-7cef-4584-960d-ab5143e6aa20 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:53.592518 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab6d7c3-249a-42da-ae5a-af280e14e2f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:53.593171 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab6d7c3-249a-42da-ae5a-af280e14e2f3 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:53.593447 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab6d7c3-249a-42da-ae5a-af280e14e2f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:53.593720 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab6d7c3-249a-42da-ae5a-af280e14e2f3 admin admin] PATH_INFO: `/v3/projects/826fa0cd5d134fcab8a9411750769a8f` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:53.597683 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab6d7c3-249a-42da-ae5a-af280e14e2f3 admin admin] RBAC: Authorizing `identity:get_project(project_id=826fa0cd5d134fcab8a9411750769a8f)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:53.600447 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab6d7c3-249a-42da-ae5a-af280e14e2f3 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:53.602588 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 140/277] 10.222.0.18 () {56 vars in 1207 bytes} [Wed Nov 2 07:16:53 2022] GET /identity/v3/projects/826fa0cd5d134fcab8a9411750769a8f => generated 302 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:16:53.603950 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b017ac-c87a-487b-8361-bcde7136d36d None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:53.604187 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b017ac-c87a-487b-8361-bcde7136d36d None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:53.604461 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b017ac-c87a-487b-8361-bcde7136d36d None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:53.605901 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 141/278] 10.222.0.18 () {54 vars in 927 bytes} [Wed Nov 2 07:16:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:16:53.611037 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c556167-667a-4e41-b8f2-488f301432af None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:53.611335 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c556167-667a-4e41-b8f2-488f301432af None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:53.611576 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c556167-667a-4e41-b8f2-488f301432af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:53.655892 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-7c556167-667a-4e41-b8f2-488f301432af None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:16:53.657012 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-7c556167-667a-4e41-b8f2-488f301432af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:16:53.674086 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 142/279] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Nov 2 07:16:53 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Nov 02 07:16:53.679782 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c556167-667a-4e41-b8f2-488f301432af None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:53.695304 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c556167-667a-4e41-b8f2-488f301432af None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:53.698030 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157595fc-6d12-4631-92c2-0e31d88dfa40 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:53.716444 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d2a8ec-55a8-483a-9121-a2393ee9f69b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:53.717127 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d2a8ec-55a8-483a-9121-a2393ee9f69b admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:53.717402 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d2a8ec-55a8-483a-9121-a2393ee9f69b admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:53.717671 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d2a8ec-55a8-483a-9121-a2393ee9f69b admin admin] PATH_INFO: `/v3/projects/826fa0cd5d134fcab8a9411750769a8f/groups/735ae239bcf0421c89bf6c19a919d4ba/roles/b494fedd00f24f0b84a529e8672a156a` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:53.720779 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d2a8ec-55a8-483a-9121-a2393ee9f69b admin admin] RBAC: Authorizing `identity:create_grant(project_id=826fa0cd5d134fcab8a9411750769a8f, group_id=735ae239bcf0421c89bf6c19a919d4ba, role_id=b494fedd00f24f0b84a529e8672a156a)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:53.723474 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d2a8ec-55a8-483a-9121-a2393ee9f69b admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:53.751674 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 143/280] 10.222.0.18 () {58 vars in 1463 bytes} [Wed Nov 2 07:16:53 2022] PUT /identity/v3/projects/826fa0cd5d134fcab8a9411750769a8f/groups/735ae239bcf0421c89bf6c19a919d4ba/roles/b494fedd00f24f0b84a529e8672a156a => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:16:53.765498 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146bf8b2-9773-4eae-b468-fefb1f437c41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:53.766578 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146bf8b2-9773-4eae-b468-fefb1f437c41 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:53.767078 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146bf8b2-9773-4eae-b468-fefb1f437c41 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:53.767543 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146bf8b2-9773-4eae-b468-fefb1f437c41 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:53.788503 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146bf8b2-9773-4eae-b468-fefb1f437c41 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:16:53.794202 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-146bf8b2-9773-4eae-b468-fefb1f437c41 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Nov 02 07:16:53.794202 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:53.794202 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:16:53.794202 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:16:53.794202 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:16:53.794202 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:16:53.794202 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:16:53.794202 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:16:53.794202 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:16:53.794202 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:16:53.794202 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:16:53.794202 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:16:53.794202 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:16:53.794202 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:16:53.794202 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:16:53.794202 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:16:53.795101 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:16:53.795101 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:16:53.795101 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:16:53.795101 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:16:53.795101 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:16:53.795101 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:16:53.795101 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:16:53.795101 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:16:53.795101 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:16:53.795101 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:16:53.795101 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:16:53.795101 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Nov 02 07:16:53.795101 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:16:53.795101 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:16:53.795101 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:16:53.795101 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:16:53.795882 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:16:53.795882 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:16:53.795882 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:16:53.795882 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:16:53.795882 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:16:53.795882 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:16:53.795882 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:16:53.795882 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:16:53.795882 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:16:53.795882 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:16:53.795882 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Nov 02 07:16:53.795882 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Nov 02 07:16:53.802736 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 138/281] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Nov 2 07:16:53 2022] GET /identity/v3/roles/reader => generated 84 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 07:16:53.805138 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d2a8ec-55a8-483a-9121-a2393ee9f69b admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:53.839270 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:53.949565 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d2a8ec-55a8-483a-9121-a2393ee9f69b admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:53.991893 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d446082-e7b7-46b4-82d0-5b22ef449d98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:53.992522 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d446082-e7b7-46b4-82d0-5b22ef449d98 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:53.992800 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d446082-e7b7-46b4-82d0-5b22ef449d98 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:53.993069 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d446082-e7b7-46b4-82d0-5b22ef449d98 None admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:53.993737 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d446082-e7b7-46b4-82d0-5b22ef449d98 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:53.994033 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d446082-e7b7-46b4-82d0-5b22ef449d98 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:53.995456 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d446082-e7b7-46b4-82d0-5b22ef449d98 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:54.004856 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 144/282] 10.222.0.18 () {56 vars in 1101 bytes} [Wed Nov 2 07:16:53 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:54.011087 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146bf8b2-9773-4eae-b468-fefb1f437c41 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:54.156105 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:54.159990 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146bf8b2-9773-4eae-b468-fefb1f437c41 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:54.174054 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d446082-e7b7-46b4-82d0-5b22ef449d98 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:54.201799 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed08ae3f-50a1-46a1-b528-6de32056002b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:54.202455 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed08ae3f-50a1-46a1-b528-6de32056002b None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:54.202661 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed08ae3f-50a1-46a1-b528-6de32056002b None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:54.202937 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed08ae3f-50a1-46a1-b528-6de32056002b None admin] PATH_INFO: `/v3/users/4bc3bad397bc4966b20b0530d141652e` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:54.204076 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed08ae3f-50a1-46a1-b528-6de32056002b None admin] RBAC: Authorizing `identity:get_user(user_id=4bc3bad397bc4966b20b0530d141652e)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:54.205534 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed08ae3f-50a1-46a1-b528-6de32056002b None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:54.207835 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 139/283] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Nov 2 07:16:54 2022] GET /identity/v3/users/4bc3bad397bc4966b20b0530d141652e => generated 293 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:54.222890 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed08ae3f-50a1-46a1-b528-6de32056002b None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:54.298600 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d446082-e7b7-46b4-82d0-5b22ef449d98 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:54.311222 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa188e5-8159-4e6b-a0dc-93e7de829214 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:54.312104 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa188e5-8159-4e6b-a0dc-93e7de829214 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:54.312517 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa188e5-8159-4e6b-a0dc-93e7de829214 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:54.312918 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa188e5-8159-4e6b-a0dc-93e7de829214 admin admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:54.327501 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa188e5-8159-4e6b-a0dc-93e7de829214 admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:16:54.332722 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-9fa188e5-8159-4e6b-a0dc-93e7de829214 admin admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Nov 02 07:16:54.332722 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:54.332722 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:16:54.332722 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:16:54.332722 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:16:54.332722 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:16:54.332722 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:16:54.332722 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:16:54.332722 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:16:54.332722 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:16:54.332722 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:16:54.332722 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:16:54.332722 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:16:54.332722 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:16:54.332722 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:16:54.332722 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:16:54.336417 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:16:54.336417 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:16:54.336417 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:16:54.336417 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:16:54.336417 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:16:54.336417 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:16:54.336417 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:16:54.336417 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:16:54.336417 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:16:54.336417 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:16:54.336417 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:16:54.336417 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:16:54.336417 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:16:54.336417 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:16:54.336417 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:16:54.336417 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:16:54.337443 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:16:54.337443 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:16:54.337443 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:16:54.337443 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:16:54.337443 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:16:54.337443 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:16:54.337443 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:16:54.337443 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:16:54.337443 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:16:54.337443 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:16:54.337443 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Nov 02 07:16:54.337443 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:16:54.337443 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 145/284] 10.222.0.18 () {56 vars in 1135 bytes} [Wed Nov 2 07:16:54 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 07:16:54.343202 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa188e5-8159-4e6b-a0dc-93e7de829214 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:54.356136 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa188e5-8159-4e6b-a0dc-93e7de829214 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:54.363203 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36c2319-6a9a-4869-bdf8-efdec5a99699 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:54.364075 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36c2319-6a9a-4869-bdf8-efdec5a99699 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:54.364617 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36c2319-6a9a-4869-bdf8-efdec5a99699 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:54.365054 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36c2319-6a9a-4869-bdf8-efdec5a99699 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:54.366521 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36c2319-6a9a-4869-bdf8-efdec5a99699 admin admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:54.367040 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36c2319-6a9a-4869-bdf8-efdec5a99699 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:54.369024 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36c2319-6a9a-4869-bdf8-efdec5a99699 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:54.377631 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed08ae3f-50a1-46a1-b528-6de32056002b None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:54.381243 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 146/285] 10.222.0.18 () {56 vars in 1132 bytes} [Wed Nov 2 07:16:54 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:16:54.383002 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8840bbe0-328c-4514-961f-44cd1d441093 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:54.383480 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8840bbe0-328c-4514-961f-44cd1d441093 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:54.383966 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8840bbe0-328c-4514-961f-44cd1d441093 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:54.385549 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 147/286] 10.222.0.18 () {54 vars in 927 bytes} [Wed Nov 2 07:16:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:16:54.389414 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8840bbe0-328c-4514-961f-44cd1d441093 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:54.406325 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8840bbe0-328c-4514-961f-44cd1d441093 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:54.413666 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c50bf36-e114-488a-a7b6-f3f352193d08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:54.413666 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c50bf36-e114-488a-a7b6-f3f352193d08 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:54.413915 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c50bf36-e114-488a-a7b6-f3f352193d08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:54.414150 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c50bf36-e114-488a-a7b6-f3f352193d08 admin admin] PATH_INFO: `/v3/groups/735ae239bcf0421c89bf6c19a919d4ba` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:54.416146 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c50bf36-e114-488a-a7b6-f3f352193d08 admin admin] RBAC: Authorizing `identity:get_group(group_id=735ae239bcf0421c89bf6c19a919d4ba)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:54.418833 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c50bf36-e114-488a-a7b6-f3f352193d08 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:54.424213 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 148/287] 10.222.0.18 () {56 vars in 1201 bytes} [Wed Nov 2 07:16:54 2022] GET /identity/v3/groups/735ae239bcf0421c89bf6c19a919d4ba => generated 226 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:16:54.424213 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900a4e86-0752-476e-8611-a94837660143 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:54.424213 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900a4e86-0752-476e-8611-a94837660143 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:54.424213 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900a4e86-0752-476e-8611-a94837660143 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:54.430491 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0dbecd-0381-447b-b7ad-cb7c2196b22d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:54.431769 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0dbecd-0381-447b-b7ad-cb7c2196b22d None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:54.432287 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0dbecd-0381-447b-b7ad-cb7c2196b22d None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:54.432776 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0dbecd-0381-447b-b7ad-cb7c2196b22d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:54.433694 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0dbecd-0381-447b-b7ad-cb7c2196b22d None admin] RBAC: Adding query filter params (role.id=979bdb3f4d8d4d97b1729d3162372a09, user.id=4bc3bad397bc4966b20b0530d141652e, scope.system=all) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:54.434232 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0dbecd-0381-447b-b7ad-cb7c2196b22d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:54.434706 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:54.436149 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0dbecd-0381-447b-b7ad-cb7c2196b22d None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:54.451221 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-900a4e86-0752-476e-8611-a94837660143 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:16:54.452486 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-900a4e86-0752-476e-8611-a94837660143 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:16:54.453310 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 140/288] 10.222.0.18 () {56 vars in 1308 bytes} [Wed Nov 2 07:16:54 2022] GET /identity/v3/role_assignments?user.id=4bc3bad397bc4966b20b0530d141652e&scope.system=all&role.id=979bdb3f4d8d4d97b1729d3162372a09 => generated 510 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:16:54.459727 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0dbecd-0381-447b-b7ad-cb7c2196b22d None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:54.464612 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 149/289] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Nov 2 07:16:54 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Nov 02 07:16:54.476013 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0dbecd-0381-447b-b7ad-cb7c2196b22d None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:54.481335 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc8c169-d073-4cd3-9668-6bb7c428c738 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:54.482212 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc8c169-d073-4cd3-9668-6bb7c428c738 admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:54.482712 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc8c169-d073-4cd3-9668-6bb7c428c738 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:54.483190 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc8c169-d073-4cd3-9668-6bb7c428c738 admin admin] PATH_INFO: `/v3/projects/826fa0cd5d134fcab8a9411750769a8f` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:54.487200 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc8c169-d073-4cd3-9668-6bb7c428c738 admin admin] RBAC: Authorizing `identity:get_project(project_id=826fa0cd5d134fcab8a9411750769a8f)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:54.489635 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc8c169-d073-4cd3-9668-6bb7c428c738 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:54.492067 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 141/290] 10.222.0.18 () {56 vars in 1207 bytes} [Wed Nov 2 07:16:54 2022] GET /identity/v3/projects/826fa0cd5d134fcab8a9411750769a8f => generated 302 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:54.497986 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900a4e86-0752-476e-8611-a94837660143 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:54.512691 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900a4e86-0752-476e-8611-a94837660143 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:54.522600 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce170b5-9f03-4b5c-aa18-68d4f395991d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:54.523394 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce170b5-9f03-4b5c-aa18-68d4f395991d admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:54.523680 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce170b5-9f03-4b5c-aa18-68d4f395991d admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:54.523956 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce170b5-9f03-4b5c-aa18-68d4f395991d admin admin] PATH_INFO: `/v3/projects/826fa0cd5d134fcab8a9411750769a8f/groups/735ae239bcf0421c89bf6c19a919d4ba/roles/8cf467a4f06c44e0b77499aeababdcc3` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:54.526229 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce170b5-9f03-4b5c-aa18-68d4f395991d admin admin] RBAC: Authorizing `identity:create_grant(project_id=826fa0cd5d134fcab8a9411750769a8f, group_id=735ae239bcf0421c89bf6c19a919d4ba, role_id=8cf467a4f06c44e0b77499aeababdcc3)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:54.528937 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce170b5-9f03-4b5c-aa18-68d4f395991d admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:54.554246 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 150/291] 10.222.0.18 () {58 vars in 1463 bytes} [Wed Nov 2 07:16:54 2022] PUT /identity/v3/projects/826fa0cd5d134fcab8a9411750769a8f/groups/735ae239bcf0421c89bf6c19a919d4ba/roles/8cf467a4f06c44e0b77499aeababdcc3 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:16:54.958066 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc8c169-d073-4cd3-9668-6bb7c428c738 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:54.973986 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc8c169-d073-4cd3-9668-6bb7c428c738 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:54.995913 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:55.041732 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7036127-ba34-481d-9bb8-1114397cc568 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:55.042384 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7036127-ba34-481d-9bb8-1114397cc568 admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:55.042679 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7036127-ba34-481d-9bb8-1114397cc568 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:55.042948 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7036127-ba34-481d-9bb8-1114397cc568 admin admin] PATH_INFO: `/v3/roles/admin` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:55.059578 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7036127-ba34-481d-9bb8-1114397cc568 admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:16:55.064393 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-b7036127-ba34-481d-9bb8-1114397cc568 admin admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 07:16:55.064393 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:55.064393 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:16:55.064393 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:16:55.064393 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:16:55.064393 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:16:55.064393 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:16:55.064393 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:16:55.064393 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:16:55.064393 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:16:55.064393 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:16:55.064393 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:16:55.064393 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:16:55.064393 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:16:55.064393 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:16:55.064393 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:16:55.065519 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:16:55.065519 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:16:55.065519 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:16:55.065519 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:16:55.065519 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:16:55.065519 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:16:55.065519 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:16:55.065519 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:16:55.065519 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:16:55.065519 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:16:55.065519 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:16:55.065519 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Nov 02 07:16:55.065519 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:16:55.065519 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:16:55.065519 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:16:55.065519 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:16:55.066302 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:16:55.066302 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:16:55.066302 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:16:55.066302 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:16:55.066302 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:16:55.066302 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:16:55.066302 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:16:55.066302 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:16:55.066302 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:16:55.066302 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:16:55.066302 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 07:16:55.066302 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Nov 02 07:16:55.070073 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 142/292] 10.222.0.18 () {56 vars in 1117 bytes} [Wed Nov 2 07:16:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 108 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:16:55.071301 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce170b5-9f03-4b5c-aa18-68d4f395991d admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:55.083771 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce170b5-9f03-4b5c-aa18-68d4f395991d admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:55.091636 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b104a22b-d17d-4332-988d-bbc880a747c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:55.092243 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b104a22b-d17d-4332-988d-bbc880a747c2 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:55.092516 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b104a22b-d17d-4332-988d-bbc880a747c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:55.092778 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b104a22b-d17d-4332-988d-bbc880a747c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:55.093430 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b104a22b-d17d-4332-988d-bbc880a747c2 admin admin] RBAC: Adding query filter params (name=admin) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:55.093725 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b104a22b-d17d-4332-988d-bbc880a747c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:55.095351 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b104a22b-d17d-4332-988d-bbc880a747c2 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:55.108200 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 151/293] 10.222.0.18 () {56 vars in 1120 bytes} [Wed Nov 2 07:16:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:55.114567 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7036127-ba34-481d-9bb8-1114397cc568 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:55.127035 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7036127-ba34-481d-9bb8-1114397cc568 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:55.131750 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda220fa-8f44-4a08-8568-27ea7bc77f93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:55.132449 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda220fa-8f44-4a08-8568-27ea7bc77f93 admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:55.132728 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda220fa-8f44-4a08-8568-27ea7bc77f93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:55.132996 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda220fa-8f44-4a08-8568-27ea7bc77f93 admin admin] PATH_INFO: `/v3/groups/21e85262c94048b4a191fac084f95d4c` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:55.142702 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda220fa-8f44-4a08-8568-27ea7bc77f93 admin admin] RBAC: Authorizing `identity:get_group(group_id=21e85262c94048b4a191fac084f95d4c)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:55.143753 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: WARNING py.warnings [None req-bda220fa-8f44-4a08-8568-27ea7bc77f93 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_group": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.group.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Nov 02 07:16:55.143753 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: warnings.warn(msg) Nov 02 07:16:55.143753 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: Nov 02 07:16:55.144957 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda220fa-8f44-4a08-8568-27ea7bc77f93 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:55.146693 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 143/294] 10.222.0.18 () {56 vars in 1201 bytes} [Wed Nov 2 07:16:55 2022] GET /identity/v3/groups/21e85262c94048b4a191fac084f95d4c => generated 229 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:55.152537 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b104a22b-d17d-4332-988d-bbc880a747c2 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:55.163235 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b104a22b-d17d-4332-988d-bbc880a747c2 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:55.170348 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddd23a4-fb77-461f-bb4f-152ef02f24be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:55.171004 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddd23a4-fb77-461f-bb4f-152ef02f24be admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:55.171280 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddd23a4-fb77-461f-bb4f-152ef02f24be admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:55.171550 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddd23a4-fb77-461f-bb4f-152ef02f24be admin admin] PATH_INFO: `/v3/projects/4ad924d40efc4e06bdd880c155ca36a4` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:55.172678 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddd23a4-fb77-461f-bb4f-152ef02f24be admin admin] RBAC: Authorizing `identity:get_project(project_id=4ad924d40efc4e06bdd880c155ca36a4)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:55.174557 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddd23a4-fb77-461f-bb4f-152ef02f24be admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:55.176328 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 152/295] 10.222.0.18 () {56 vars in 1207 bytes} [Wed Nov 2 07:16:55 2022] GET /identity/v3/projects/4ad924d40efc4e06bdd880c155ca36a4 => generated 344 bytes in 24 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:55.182631 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda220fa-8f44-4a08-8568-27ea7bc77f93 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:55.195430 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda220fa-8f44-4a08-8568-27ea7bc77f93 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:55.200038 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0de2e0-fb4f-4bae-bdc5-cbcfddc8a4e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:55.200694 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0de2e0-fb4f-4bae-bdc5-cbcfddc8a4e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:55.200971 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0de2e0-fb4f-4bae-bdc5-cbcfddc8a4e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:55.201236 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0de2e0-fb4f-4bae-bdc5-cbcfddc8a4e9 admin admin] PATH_INFO: `/v3/projects/4ad924d40efc4e06bdd880c155ca36a4/groups/21e85262c94048b4a191fac084f95d4c/roles/c4bf019fc6c44c71a6c984f6d0bab75f` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:55.204338 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0de2e0-fb4f-4bae-bdc5-cbcfddc8a4e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ad924d40efc4e06bdd880c155ca36a4, group_id=21e85262c94048b4a191fac084f95d4c, role_id=c4bf019fc6c44c71a6c984f6d0bab75f)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:55.206991 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0de2e0-fb4f-4bae-bdc5-cbcfddc8a4e9 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:55.233706 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 144/296] 10.222.0.18 () {58 vars in 1463 bytes} [Wed Nov 2 07:16:55 2022] PUT /identity/v3/projects/4ad924d40efc4e06bdd880c155ca36a4/groups/21e85262c94048b4a191fac084f95d4c/roles/c4bf019fc6c44c71a6c984f6d0bab75f => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:16:55.577529 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa698e9-d14a-47d8-853b-81bac3148df5 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:55.578229 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa698e9-d14a-47d8-853b-81bac3148df5 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:55.578229 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa698e9-d14a-47d8-853b-81bac3148df5 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:55.579573 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 153/297] 10.222.0.18 () {54 vars in 927 bytes} [Wed Nov 2 07:16:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:16:55.585714 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd06bbcf-917a-40fe-8264-1b5a55f99b52 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:55.586211 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd06bbcf-917a-40fe-8264-1b5a55f99b52 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:55.586382 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd06bbcf-917a-40fe-8264-1b5a55f99b52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:55.606867 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-cd06bbcf-917a-40fe-8264-1b5a55f99b52 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:16:55.608128 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-cd06bbcf-917a-40fe-8264-1b5a55f99b52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:16:55.703221 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 145/298] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Nov 2 07:16:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:16:55.790712 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad7bb0d-f64c-4bc6-bf6e-40f50cc78784 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:55.791221 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad7bb0d-f64c-4bc6-bf6e-40f50cc78784 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:55.791636 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad7bb0d-f64c-4bc6-bf6e-40f50cc78784 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:55.793137 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 154/299] 10.222.0.18 () {54 vars in 927 bytes} [Wed Nov 2 07:16:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:16:55.798676 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9d778b-bb83-434c-aad4-4825a237eead None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:55.799123 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9d778b-bb83-434c-aad4-4825a237eead None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:55.799598 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9d778b-bb83-434c-aad4-4825a237eead None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:55.820538 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-5e9d778b-bb83-434c-aad4-4825a237eead None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:16:55.822001 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-5e9d778b-bb83-434c-aad4-4825a237eead None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:16:55.899669 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 146/300] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Nov 2 07:16:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:16:56.150339 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad7bb0d-f64c-4bc6-bf6e-40f50cc78784 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:56.269806 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad7bb0d-f64c-4bc6-bf6e-40f50cc78784 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:56.313226 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec1c907-39e1-4ae7-8b8b-bb2a56b8eadc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:56.313910 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec1c907-39e1-4ae7-8b8b-bb2a56b8eadc None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:56.314190 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec1c907-39e1-4ae7-8b8b-bb2a56b8eadc None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:56.314480 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec1c907-39e1-4ae7-8b8b-bb2a56b8eadc None admin] PATH_INFO: `/v3/roles/member` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:56.334029 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec1c907-39e1-4ae7-8b8b-bb2a56b8eadc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:16:56.339476 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-bec1c907-39e1-4ae7-8b8b-bb2a56b8eadc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Nov 02 07:16:56.339476 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:56.339476 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:16:56.339476 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:16:56.339476 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:16:56.339476 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:16:56.339476 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:16:56.339476 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:16:56.339476 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:16:56.339476 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:16:56.339476 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:16:56.339476 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:16:56.339476 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:16:56.339476 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:16:56.339476 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:16:56.339476 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:16:56.340745 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:16:56.340745 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:16:56.340745 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:16:56.340745 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:16:56.340745 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:16:56.340745 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:16:56.340745 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:16:56.340745 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:16:56.340745 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:16:56.340745 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:16:56.340745 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:16:56.340745 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:16:56.340745 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:16:56.340745 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:16:56.340745 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:16:56.340745 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:16:56.341882 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:16:56.341882 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:16:56.341882 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:16:56.341882 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:16:56.341882 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:16:56.341882 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:16:56.341882 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:16:56.341882 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:16:56.341882 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:16:56.341882 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:16:56.341882 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Nov 02 07:16:56.341882 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:16:56.341882 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 155/301] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Nov 2 07:16:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:16:56.348030 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9d778b-bb83-434c-aad4-4825a237eead None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:56.351759 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:56.382786 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec1c907-39e1-4ae7-8b8b-bb2a56b8eadc None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:56.429326 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:56.482338 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9d778b-bb83-434c-aad4-4825a237eead None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:56.525393 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060fc5f8-31bf-47db-ad1c-474eb1b1f5de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:56.526344 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060fc5f8-31bf-47db-ad1c-474eb1b1f5de None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:56.526863 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060fc5f8-31bf-47db-ad1c-474eb1b1f5de None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:56.527981 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060fc5f8-31bf-47db-ad1c-474eb1b1f5de None admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:56.528877 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060fc5f8-31bf-47db-ad1c-474eb1b1f5de None admin] RBAC: Adding query filter params (name=member) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:56.529383 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060fc5f8-31bf-47db-ad1c-474eb1b1f5de None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:56.531467 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060fc5f8-31bf-47db-ad1c-474eb1b1f5de None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:56.545852 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 147/302] 10.222.0.18 () {56 vars in 1101 bytes} [Wed Nov 2 07:16:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:56.551401 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060fc5f8-31bf-47db-ad1c-474eb1b1f5de None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:56.598309 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec1c907-39e1-4ae7-8b8b-bb2a56b8eadc None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:56.635946 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:56.675061 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185ec826-3a64-41a0-bc3a-ad19468919fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:56.676004 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185ec826-3a64-41a0-bc3a-ad19468919fb None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:56.676495 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185ec826-3a64-41a0-bc3a-ad19468919fb None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:56.676991 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185ec826-3a64-41a0-bc3a-ad19468919fb None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:56.684321 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060fc5f8-31bf-47db-ad1c-474eb1b1f5de None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:56.709407 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185ec826-3a64-41a0-bc3a-ad19468919fb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:16:56.715188 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-185ec826-3a64-41a0-bc3a-ad19468919fb None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Nov 02 07:16:56.715188 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:56.715188 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:16:56.715188 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:16:56.715188 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:16:56.715188 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:16:56.715188 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:16:56.715188 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:16:56.715188 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:16:56.715188 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:16:56.715188 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:16:56.715188 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:16:56.715188 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:16:56.715188 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:16:56.715188 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:16:56.715188 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:16:56.716059 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:16:56.716059 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:16:56.716059 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:16:56.716059 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:16:56.716059 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:16:56.716059 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:16:56.716059 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:16:56.716059 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:16:56.716059 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:16:56.716059 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:16:56.716059 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:16:56.716059 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:16:56.716059 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:16:56.716059 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:16:56.716059 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:16:56.716059 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:16:56.716842 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:16:56.716842 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:16:56.716842 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:16:56.716842 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:16:56.716842 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:16:56.716842 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:16:56.716842 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:16:56.716842 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:16:56.716842 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:16:56.716842 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:16:56.716842 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Nov 02 07:16:56.716842 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:16:56.723461 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 156/303] 10.222.0.18 () {56 vars in 1114 bytes} [Wed Nov 2 07:16:56 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 341 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:16:56.729239 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd66c7da-421b-4b12-b832-d81f2b48dd17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:56.729854 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd66c7da-421b-4b12-b832-d81f2b48dd17 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:56.730138 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd66c7da-421b-4b12-b832-d81f2b48dd17 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:56.730589 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd66c7da-421b-4b12-b832-d81f2b48dd17 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:56.750564 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185ec826-3a64-41a0-bc3a-ad19468919fb None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:56.880190 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:56.886772 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd66c7da-421b-4b12-b832-d81f2b48dd17 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:56.888376 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd66c7da-421b-4b12-b832-d81f2b48dd17 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:56.891874 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185ec826-3a64-41a0-bc3a-ad19468919fb None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:56.983209 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35faaa0-ec3e-43b8-8438-124c7d0b8eb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:56.983752 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35faaa0-ec3e-43b8-8438-124c7d0b8eb0 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:56.984038 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35faaa0-ec3e-43b8-8438-124c7d0b8eb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:56.984311 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35faaa0-ec3e-43b8-8438-124c7d0b8eb0 None admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:56.984984 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35faaa0-ec3e-43b8-8438-124c7d0b8eb0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:56.985284 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35faaa0-ec3e-43b8-8438-124c7d0b8eb0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:56.986698 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35faaa0-ec3e-43b8-8438-124c7d0b8eb0 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:56.986900 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:56.996107 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 157/304] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Nov 2 07:16:56 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:57.001850 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35faaa0-ec3e-43b8-8438-124c7d0b8eb0 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:57.035858 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 148/305] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Nov 2 07:16:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 485 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Nov 02 07:16:57.043335 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd66c7da-421b-4b12-b832-d81f2b48dd17 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:57.109324 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35faaa0-ec3e-43b8-8438-124c7d0b8eb0 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:57.147463 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24b17c2-56f4-4c36-b8a9-62f0537e4581 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:57.148075 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24b17c2-56f4-4c36-b8a9-62f0537e4581 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:57.148357 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24b17c2-56f4-4c36-b8a9-62f0537e4581 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:57.148682 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24b17c2-56f4-4c36-b8a9-62f0537e4581 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:57.152489 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd66c7da-421b-4b12-b832-d81f2b48dd17 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:57.173526 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:57.204915 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d957586b-85e8-4587-a331-6f9b81a1cd3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:57.205952 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d957586b-85e8-4587-a331-6f9b81a1cd3d None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:57.206453 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d957586b-85e8-4587-a331-6f9b81a1cd3d None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:57.206921 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d957586b-85e8-4587-a331-6f9b81a1cd3d None admin] PATH_INFO: `/v3/projects/a6b063398180486c85fc258b5a513046` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:57.209703 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d957586b-85e8-4587-a331-6f9b81a1cd3d None admin] RBAC: Authorizing `identity:get_project(project_id=a6b063398180486c85fc258b5a513046)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:57.212138 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d957586b-85e8-4587-a331-6f9b81a1cd3d None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:57.214008 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 149/306] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Nov 2 07:16:57 2022] GET /identity/v3/projects/a6b063398180486c85fc258b5a513046 => generated 298 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:57.219748 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d957586b-85e8-4587-a331-6f9b81a1cd3d None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:57.222042 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24b17c2-56f4-4c36-b8a9-62f0537e4581 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:57.223570 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24b17c2-56f4-4c36-b8a9-62f0537e4581 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:57.275107 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:57.326929 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 158/307] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Nov 2 07:16:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Nov 02 07:16:57.332637 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24b17c2-56f4-4c36-b8a9-62f0537e4581 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:57.394738 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:57.413624 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d957586b-85e8-4587-a331-6f9b81a1cd3d None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:57.438963 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24b17c2-56f4-4c36-b8a9-62f0537e4581 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:57.453535 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a2a970-7965-4f6e-bb18-90113b8312dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:57.454625 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a2a970-7965-4f6e-bb18-90113b8312dc None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:57.455127 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a2a970-7965-4f6e-bb18-90113b8312dc None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:57.455598 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a2a970-7965-4f6e-bb18-90113b8312dc None admin] PATH_INFO: `/v3/groups/735ae239bcf0421c89bf6c19a919d4ba` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:57.457077 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a2a970-7965-4f6e-bb18-90113b8312dc None admin] RBAC: Authorizing `identity:get_group(group_id=735ae239bcf0421c89bf6c19a919d4ba)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:57.458927 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a2a970-7965-4f6e-bb18-90113b8312dc None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:57.461363 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 150/308] 10.222.0.18 () {56 vars in 1180 bytes} [Wed Nov 2 07:16:57 2022] GET /identity/v3/groups/735ae239bcf0421c89bf6c19a919d4ba => generated 226 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:16:57.467729 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a2a970-7965-4f6e-bb18-90113b8312dc None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:57.512160 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08691fc-6b4a-43a3-afea-44be02a7d888 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:57.512370 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08691fc-6b4a-43a3-afea-44be02a7d888 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:57.512628 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08691fc-6b4a-43a3-afea-44be02a7d888 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:57.512808 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08691fc-6b4a-43a3-afea-44be02a7d888 None admin] PATH_INFO: `/v3/projects/a6b063398180486c85fc258b5a513046` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:57.513927 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08691fc-6b4a-43a3-afea-44be02a7d888 None admin] RBAC: Authorizing `identity:get_project(project_id=a6b063398180486c85fc258b5a513046)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:57.515455 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08691fc-6b4a-43a3-afea-44be02a7d888 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:57.517149 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 159/309] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Nov 2 07:16:57 2022] GET /identity/v3/projects/a6b063398180486c85fc258b5a513046 => generated 298 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:16:57.524612 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08691fc-6b4a-43a3-afea-44be02a7d888 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:57.585783 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:57.590941 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a2a970-7965-4f6e-bb18-90113b8312dc None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:57.637830 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72a88e3-0993-4e1b-8b75-10604a08f04e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:57.638522 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72a88e3-0993-4e1b-8b75-10604a08f04e None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:57.638857 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72a88e3-0993-4e1b-8b75-10604a08f04e None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:57.639020 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72a88e3-0993-4e1b-8b75-10604a08f04e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:57.639764 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72a88e3-0993-4e1b-8b75-10604a08f04e None admin] RBAC: Adding query filter params (group.id=735ae239bcf0421c89bf6c19a919d4ba, role.id=b494fedd00f24f0b84a529e8672a156a, scope.project.id=a6b063398180486c85fc258b5a513046) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:57.640039 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72a88e3-0993-4e1b-8b75-10604a08f04e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:57.645415 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72a88e3-0993-4e1b-8b75-10604a08f04e None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:57.659077 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 151/310] 10.222.0.18 () {56 vars in 1376 bytes} [Wed Nov 2 07:16:57 2022] GET /identity/v3/role_assignments?group.id=735ae239bcf0421c89bf6c19a919d4ba&scope.project.id=a6b063398180486c85fc258b5a513046&role.id=b494fedd00f24f0b84a529e8672a156a => generated 611 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:57.660191 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3192230c-6543-40a6-8dbf-4969f54e49b7 None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:57.660474 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3192230c-6543-40a6-8dbf-4969f54e49b7 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:57.660747 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3192230c-6543-40a6-8dbf-4969f54e49b7 None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:57.663840 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 152/311] 10.222.0.18 () {54 vars in 927 bytes} [Wed Nov 2 07:16:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:16:57.665292 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd1a708-7150-4c2e-9f50-c28b95a13f2d None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:57.665571 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd1a708-7150-4c2e-9f50-c28b95a13f2d None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:57.665843 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd1a708-7150-4c2e-9f50-c28b95a13f2d None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:57.667550 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 153/312] 10.222.0.18 () {54 vars in 927 bytes} [Wed Nov 2 07:16:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:16:57.668742 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b52fc77-9fd5-4cf2-a6e5-3f9c14bd4eea None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:57.669109 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b52fc77-9fd5-4cf2-a6e5-3f9c14bd4eea None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:57.669382 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b52fc77-9fd5-4cf2-a6e5-3f9c14bd4eea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:57.690016 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-0b52fc77-9fd5-4cf2-a6e5-3f9c14bd4eea None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:16:57.691343 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-0b52fc77-9fd5-4cf2-a6e5-3f9c14bd4eea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:16:57.701204 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08691fc-6b4a-43a3-afea-44be02a7d888 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:57.738608 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:57.755227 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb6b0fd-7306-497f-8dab-69549a6b9a15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:57.756625 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb6b0fd-7306-497f-8dab-69549a6b9a15 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:57.757599 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb6b0fd-7306-497f-8dab-69549a6b9a15 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:57.758085 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb6b0fd-7306-497f-8dab-69549a6b9a15 None admin] PATH_INFO: `/v3/groups/735ae239bcf0421c89bf6c19a919d4ba` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:57.759596 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb6b0fd-7306-497f-8dab-69549a6b9a15 None admin] RBAC: Authorizing `identity:get_group(group_id=735ae239bcf0421c89bf6c19a919d4ba)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:57.762456 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb6b0fd-7306-497f-8dab-69549a6b9a15 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:57.765688 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 160/313] 10.222.0.18 () {56 vars in 1180 bytes} [Wed Nov 2 07:16:57 2022] GET /identity/v3/groups/735ae239bcf0421c89bf6c19a919d4ba => generated 226 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:57.767286 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ef3567-2e6a-4e8c-94c9-d739c429c4e5 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:57.767770 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ef3567-2e6a-4e8c-94c9-d739c429c4e5 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:57.768233 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ef3567-2e6a-4e8c-94c9-d739c429c4e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:57.782325 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 154/314] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Nov 2 07:16:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:16:57.784079 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b52fc77-9fd5-4cf2-a6e5-3f9c14bd4eea None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:57.792778 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-69ef3567-2e6a-4e8c-94c9-d739c429c4e5 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:16:57.795095 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-69ef3567-2e6a-4e8c-94c9-d739c429c4e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:16:57.887659 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 161/315] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Nov 2 07:16:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:16:57.940760 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:57.948874 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b52fc77-9fd5-4cf2-a6e5-3f9c14bd4eea None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:57.989823 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af85fb8-18d7-4ed8-ac3a-ad5a09d2a9a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:57.991598 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af85fb8-18d7-4ed8-ac3a-ad5a09d2a9a2 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:57.992122 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af85fb8-18d7-4ed8-ac3a-ad5a09d2a9a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:57.992581 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af85fb8-18d7-4ed8-ac3a-ad5a09d2a9a2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:57.993520 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af85fb8-18d7-4ed8-ac3a-ad5a09d2a9a2 None admin] RBAC: Adding query filter params (group.id=735ae239bcf0421c89bf6c19a919d4ba, role.id=8cf467a4f06c44e0b77499aeababdcc3, scope.project.id=a6b063398180486c85fc258b5a513046) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:57.994028 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af85fb8-18d7-4ed8-ac3a-ad5a09d2a9a2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:57.995782 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af85fb8-18d7-4ed8-ac3a-ad5a09d2a9a2 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:58.002132 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 155/316] 10.222.0.18 () {56 vars in 1376 bytes} [Wed Nov 2 07:16:57 2022] GET /identity/v3/role_assignments?group.id=735ae239bcf0421c89bf6c19a919d4ba&scope.project.id=a6b063398180486c85fc258b5a513046&role.id=8cf467a4f06c44e0b77499aeababdcc3 => generated 611 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:58.114335 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ef3567-2e6a-4e8c-94c9-d739c429c4e5 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:58.208195 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af85fb8-18d7-4ed8-ac3a-ad5a09d2a9a2 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:58.258582 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ef3567-2e6a-4e8c-94c9-d739c429c4e5 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:58.287954 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:58.302918 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81acf4d4-d011-424a-924a-26921208018c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:58.303834 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81acf4d4-d011-424a-924a-26921208018c None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:58.304114 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81acf4d4-d011-424a-924a-26921208018c None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:58.304382 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81acf4d4-d011-424a-924a-26921208018c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:58.316645 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81acf4d4-d011-424a-924a-26921208018c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:16:58.322425 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-81acf4d4-d011-424a-924a-26921208018c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Nov 02 07:16:58.322425 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:58.322425 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:16:58.322425 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:16:58.322425 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:16:58.322425 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:16:58.322425 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:16:58.322425 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:16:58.322425 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:16:58.322425 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:16:58.322425 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:16:58.322425 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:16:58.322425 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:16:58.322425 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:16:58.322425 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:16:58.322425 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:16:58.323321 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:16:58.323321 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:16:58.323321 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:16:58.323321 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:16:58.323321 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:16:58.323321 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:16:58.323321 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:16:58.323321 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:16:58.323321 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:16:58.323321 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:16:58.323321 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:16:58.323321 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:16:58.323321 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:16:58.323321 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:16:58.323321 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:16:58.323321 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:16:58.324112 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:16:58.324112 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:16:58.324112 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:16:58.324112 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:16:58.324112 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:16:58.324112 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:16:58.324112 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:16:58.324112 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:16:58.324112 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:16:58.324112 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:16:58.324112 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Nov 02 07:16:58.324112 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:16:58.324112 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 162/317] 10.222.0.18 () {56 vars in 1114 bytes} [Wed Nov 2 07:16:58 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:16:58.324752 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591765fc-c528-4657-aa17-538ddf487a9b None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:58.324965 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591765fc-c528-4657-aa17-538ddf487a9b None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:58.325236 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591765fc-c528-4657-aa17-538ddf487a9b None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:58.327000 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 163/318] 10.222.0.18 () {54 vars in 927 bytes} [Wed Nov 2 07:16:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:16:58.332151 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591765fc-c528-4657-aa17-538ddf487a9b None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:58.349487 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af85fb8-18d7-4ed8-ac3a-ad5a09d2a9a2 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:58.391014 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da51e1e-fc9c-4819-b5b1-101d3e0cc99b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:58.391648 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da51e1e-fc9c-4819-b5b1-101d3e0cc99b None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:58.391926 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da51e1e-fc9c-4819-b5b1-101d3e0cc99b None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:58.392195 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da51e1e-fc9c-4819-b5b1-101d3e0cc99b None admin] PATH_INFO: `/v3/roles/member` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:58.402870 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da51e1e-fc9c-4819-b5b1-101d3e0cc99b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:16:58.407790 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-4da51e1e-fc9c-4819-b5b1-101d3e0cc99b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Nov 02 07:16:58.407790 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:58.407790 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:16:58.407790 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:16:58.407790 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:16:58.407790 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:16:58.407790 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:16:58.407790 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:16:58.407790 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:16:58.407790 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:16:58.407790 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:16:58.407790 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:16:58.407790 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:16:58.407790 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:16:58.407790 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:16:58.407790 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:16:58.408723 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:16:58.408723 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:16:58.408723 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:16:58.408723 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:16:58.408723 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:16:58.408723 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:16:58.408723 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:16:58.408723 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:16:58.408723 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:16:58.408723 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:16:58.408723 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:16:58.408723 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Nov 02 07:16:58.408723 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:16:58.408723 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:16:58.408723 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:16:58.408723 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:16:58.409501 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:16:58.409501 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:16:58.409501 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:16:58.409501 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:16:58.409501 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:16:58.409501 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:16:58.409501 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:16:58.409501 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:16:58.409501 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:16:58.409501 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:16:58.409501 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Nov 02 07:16:58.409501 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Nov 02 07:16:58.409501 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 156/319] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Nov 2 07:16:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 07:16:58.410716 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc2b872-c132-4c91-9015-d70941be4339 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:58.410994 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc2b872-c132-4c91-9015-d70941be4339 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:58.411264 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc2b872-c132-4c91-9015-d70941be4339 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:58.431679 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-4dc2b872-c132-4c91-9015-d70941be4339 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:16:58.433219 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-4dc2b872-c132-4c91-9015-d70941be4339 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:16:58.450627 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:58.463878 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591765fc-c528-4657-aa17-538ddf487a9b None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:58.508855 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ba0efa-36be-4662-9f60-20abeb302050 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:58.509463 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ba0efa-36be-4662-9f60-20abeb302050 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:58.509739 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ba0efa-36be-4662-9f60-20abeb302050 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:58.510007 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ba0efa-36be-4662-9f60-20abeb302050 None admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:58.510697 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ba0efa-36be-4662-9f60-20abeb302050 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:58.510992 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ba0efa-36be-4662-9f60-20abeb302050 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:58.512345 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ba0efa-36be-4662-9f60-20abeb302050 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:58.513816 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 157/320] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Nov 2 07:16:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:16:58.514699 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc2b872-c132-4c91-9015-d70941be4339 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:58.526386 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 164/321] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Nov 2 07:16:58 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:58.532210 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ba0efa-36be-4662-9f60-20abeb302050 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:58.591996 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:58.638779 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc2b872-c132-4c91-9015-d70941be4339 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:58.690112 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ba0efa-36be-4662-9f60-20abeb302050 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:58.694018 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeab5b38-05ef-472e-9656-9199d3c9263e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:58.695316 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeab5b38-05ef-472e-9656-9199d3c9263e None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:58.695316 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeab5b38-05ef-472e-9656-9199d3c9263e None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:58.695316 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeab5b38-05ef-472e-9656-9199d3c9263e None admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:58.695841 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeab5b38-05ef-472e-9656-9199d3c9263e None admin] RBAC: Adding query filter params (name=member) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:58.696138 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeab5b38-05ef-472e-9656-9199d3c9263e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:58.697512 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeab5b38-05ef-472e-9656-9199d3c9263e None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:58.710461 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 158/322] 10.222.0.18 () {56 vars in 1101 bytes} [Wed Nov 2 07:16:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:58.715886 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeab5b38-05ef-472e-9656-9199d3c9263e None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:58.734066 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:58.743841 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3e646a-78d1-48bb-a88f-9fcdb916d5a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:58.744455 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3e646a-78d1-48bb-a88f-9fcdb916d5a7 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:58.744733 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3e646a-78d1-48bb-a88f-9fcdb916d5a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:58.745010 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3e646a-78d1-48bb-a88f-9fcdb916d5a7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:58.834004 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3e646a-78d1-48bb-a88f-9fcdb916d5a7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:58.835611 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3e646a-78d1-48bb-a88f-9fcdb916d5a7 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:58.853522 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeab5b38-05ef-472e-9656-9199d3c9263e None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:58.890383 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:58.893832 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b70ebdc-73f4-4208-8cd9-35fa92292907 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:58.895216 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b70ebdc-73f4-4208-8cd9-35fa92292907 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:58.895216 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b70ebdc-73f4-4208-8cd9-35fa92292907 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:58.895556 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b70ebdc-73f4-4208-8cd9-35fa92292907 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:58.957523 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 165/323] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Nov 2 07:16:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Nov 02 07:16:58.958642 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3e646a-78d1-48bb-a88f-9fcdb916d5a7 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:58.984360 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b70ebdc-73f4-4208-8cd9-35fa92292907 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:58.986053 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b70ebdc-73f4-4208-8cd9-35fa92292907 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:58.997777 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:59.081234 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3e646a-78d1-48bb-a88f-9fcdb916d5a7 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:59.108894 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 159/324] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Nov 2 07:16:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Nov 02 07:16:59.109749 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b70ebdc-73f4-4208-8cd9-35fa92292907 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:59.120812 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d93eb8-bdb0-4f18-a7e9-41932db7c1bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:59.121454 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d93eb8-bdb0-4f18-a7e9-41932db7c1bd None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:59.121732 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d93eb8-bdb0-4f18-a7e9-41932db7c1bd None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:59.122022 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d93eb8-bdb0-4f18-a7e9-41932db7c1bd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:59.135315 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d93eb8-bdb0-4f18-a7e9-41932db7c1bd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:16:59.140284 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-12d93eb8-bdb0-4f18-a7e9-41932db7c1bd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 07:16:59.140284 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:59.140284 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:16:59.140284 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:16:59.140284 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:16:59.140284 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:16:59.140284 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:16:59.140284 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:16:59.140284 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:16:59.140284 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:16:59.140284 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:16:59.140284 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:16:59.140284 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:16:59.140284 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:16:59.140284 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:16:59.140284 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:16:59.141147 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:16:59.141147 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:16:59.141147 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:16:59.141147 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:16:59.141147 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:16:59.141147 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:16:59.141147 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:16:59.141147 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:16:59.141147 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:16:59.141147 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:16:59.141147 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:16:59.141147 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:16:59.141147 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:16:59.141147 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:16:59.141147 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:16:59.141147 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:16:59.141933 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:16:59.141933 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:16:59.141933 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:16:59.141933 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:16:59.141933 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:16:59.141933 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:16:59.141933 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:16:59.141933 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:16:59.141933 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:16:59.141933 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:16:59.141933 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 07:16:59.141933 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:16:59.141933 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 166/325] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Nov 2 07:16:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:16:59.142671 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d93eb8-bdb0-4f18-a7e9-41932db7c1bd None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:59.146276 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:59.232245 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b70ebdc-73f4-4208-8cd9-35fa92292907 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:59.245140 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:59.270644 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d93eb8-bdb0-4f18-a7e9-41932db7c1bd None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:59.273057 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0689b5e3-3def-49e9-91af-08f7f94930a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:59.273696 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0689b5e3-3def-49e9-91af-08f7f94930a5 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:59.273978 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0689b5e3-3def-49e9-91af-08f7f94930a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:59.274251 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0689b5e3-3def-49e9-91af-08f7f94930a5 None admin] PATH_INFO: `/v3/projects/826fa0cd5d134fcab8a9411750769a8f` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:59.275404 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0689b5e3-3def-49e9-91af-08f7f94930a5 None admin] RBAC: Authorizing `identity:get_project(project_id=826fa0cd5d134fcab8a9411750769a8f)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:59.276844 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0689b5e3-3def-49e9-91af-08f7f94930a5 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:59.278492 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 160/326] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Nov 2 07:16:59 2022] GET /identity/v3/projects/826fa0cd5d134fcab8a9411750769a8f => generated 302 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:59.279254 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0689b5e3-3def-49e9-91af-08f7f94930a5 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:59.305714 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1ab6d3-2539-40dd-9914-6f377fd29552 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:59.306509 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1ab6d3-2539-40dd-9914-6f377fd29552 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:59.306856 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1ab6d3-2539-40dd-9914-6f377fd29552 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:59.307193 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1ab6d3-2539-40dd-9914-6f377fd29552 None admin] PATH_INFO: `/v3/projects/826fa0cd5d134fcab8a9411750769a8f` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:59.308363 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1ab6d3-2539-40dd-9914-6f377fd29552 None admin] RBAC: Authorizing `identity:get_project(project_id=826fa0cd5d134fcab8a9411750769a8f)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:59.309837 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1ab6d3-2539-40dd-9914-6f377fd29552 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:59.311488 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 167/327] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Nov 2 07:16:59 2022] GET /identity/v3/projects/826fa0cd5d134fcab8a9411750769a8f => generated 302 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:59.312244 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1ab6d3-2539-40dd-9914-6f377fd29552 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:59.379993 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0689b5e3-3def-49e9-91af-08f7f94930a5 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:59.380713 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:59.411339 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1ab6d3-2539-40dd-9914-6f377fd29552 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:59.413991 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913d3614-43bb-48f9-a41d-d8c4a9bc8ae1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:59.414641 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913d3614-43bb-48f9-a41d-d8c4a9bc8ae1 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:59.414921 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913d3614-43bb-48f9-a41d-d8c4a9bc8ae1 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:59.415199 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913d3614-43bb-48f9-a41d-d8c4a9bc8ae1 None admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:59.415843 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913d3614-43bb-48f9-a41d-d8c4a9bc8ae1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:59.416138 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913d3614-43bb-48f9-a41d-d8c4a9bc8ae1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:59.417540 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913d3614-43bb-48f9-a41d-d8c4a9bc8ae1 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:59.425535 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 161/328] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Nov 2 07:16:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:59.426307 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913d3614-43bb-48f9-a41d-d8c4a9bc8ae1 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:59.446403 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0d0ecc-bba4-4fdf-8a9b-163b1dac3859 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:59.447034 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0d0ecc-bba4-4fdf-8a9b-163b1dac3859 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:59.447314 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0d0ecc-bba4-4fdf-8a9b-163b1dac3859 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:59.447582 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0d0ecc-bba4-4fdf-8a9b-163b1dac3859 None admin] PATH_INFO: `/v3/groups/735ae239bcf0421c89bf6c19a919d4ba` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:59.448678 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0d0ecc-bba4-4fdf-8a9b-163b1dac3859 None admin] RBAC: Authorizing `identity:get_group(group_id=735ae239bcf0421c89bf6c19a919d4ba)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:59.450121 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0d0ecc-bba4-4fdf-8a9b-163b1dac3859 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:59.451698 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 168/329] 10.222.0.18 () {56 vars in 1180 bytes} [Wed Nov 2 07:16:59 2022] GET /identity/v3/groups/735ae239bcf0421c89bf6c19a919d4ba => generated 226 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:59.452452 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0d0ecc-bba4-4fdf-8a9b-163b1dac3859 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:59.468268 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:59.527851 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913d3614-43bb-48f9-a41d-d8c4a9bc8ae1 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:59.550634 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0d0ecc-bba4-4fdf-8a9b-163b1dac3859 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:59.560546 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e1692e-7c08-4bdf-b0d0-22e48ceaf553 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:59.561205 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e1692e-7c08-4bdf-b0d0-22e48ceaf553 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:59.561491 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e1692e-7c08-4bdf-b0d0-22e48ceaf553 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:59.561763 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e1692e-7c08-4bdf-b0d0-22e48ceaf553 None admin] PATH_INFO: `/v3/groups/735ae239bcf0421c89bf6c19a919d4ba` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:59.562887 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e1692e-7c08-4bdf-b0d0-22e48ceaf553 None admin] RBAC: Authorizing `identity:get_group(group_id=735ae239bcf0421c89bf6c19a919d4ba)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:59.564304 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e1692e-7c08-4bdf-b0d0-22e48ceaf553 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:59.565855 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 162/330] 10.222.0.18 () {56 vars in 1180 bytes} [Wed Nov 2 07:16:59 2022] GET /identity/v3/groups/735ae239bcf0421c89bf6c19a919d4ba => generated 226 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:59.566635 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e1692e-7c08-4bdf-b0d0-22e48ceaf553 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:59.588794 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84d8d7b-2f64-4bc7-b6e2-ddc513231e15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:59.589398 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84d8d7b-2f64-4bc7-b6e2-ddc513231e15 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:59.589676 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84d8d7b-2f64-4bc7-b6e2-ddc513231e15 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:59.589947 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84d8d7b-2f64-4bc7-b6e2-ddc513231e15 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:59.597834 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:59.658270 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84d8d7b-2f64-4bc7-b6e2-ddc513231e15 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:59.659761 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84d8d7b-2f64-4bc7-b6e2-ddc513231e15 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:59.665121 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e1692e-7c08-4bdf-b0d0-22e48ceaf553 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:59.673867 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:59.699544 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47dc109f-046d-49d4-a3ef-cda349d73249 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:59.700146 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dc109f-046d-49d4-a3ef-cda349d73249 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:59.700424 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dc109f-046d-49d4-a3ef-cda349d73249 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:59.700690 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dc109f-046d-49d4-a3ef-cda349d73249 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:59.701398 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47dc109f-046d-49d4-a3ef-cda349d73249 None admin] RBAC: Adding query filter params (group.id=735ae239bcf0421c89bf6c19a919d4ba, role.id=8cf467a4f06c44e0b77499aeababdcc3, scope.project.id=826fa0cd5d134fcab8a9411750769a8f) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:59.701698 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47dc109f-046d-49d4-a3ef-cda349d73249 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:59.703131 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47dc109f-046d-49d4-a3ef-cda349d73249 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:59.708758 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 163/331] 10.222.0.18 () {56 vars in 1376 bytes} [Wed Nov 2 07:16:59 2022] GET /identity/v3/role_assignments?group.id=735ae239bcf0421c89bf6c19a919d4ba&scope.project.id=826fa0cd5d134fcab8a9411750769a8f&role.id=8cf467a4f06c44e0b77499aeababdcc3 => generated 611 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:59.710672 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47dc109f-046d-49d4-a3ef-cda349d73249 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:59.770407 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 169/332] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Nov 2 07:16:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Nov 02 07:16:59.776584 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84d8d7b-2f64-4bc7-b6e2-ddc513231e15 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:59.820023 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:59.834013 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47dc109f-046d-49d4-a3ef-cda349d73249 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:59.873252 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff2357c-2c58-4261-88cb-8c52aa69e0a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:59.874261 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff2357c-2c58-4261-88cb-8c52aa69e0a6 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:59.874578 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff2357c-2c58-4261-88cb-8c52aa69e0a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:59.874963 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff2357c-2c58-4261-88cb-8c52aa69e0a6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:59.875587 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff2357c-2c58-4261-88cb-8c52aa69e0a6 None admin] RBAC: Adding query filter params (group.id=735ae239bcf0421c89bf6c19a919d4ba, role.id=b494fedd00f24f0b84a529e8672a156a, scope.project.id=826fa0cd5d134fcab8a9411750769a8f) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:59.875892 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff2357c-2c58-4261-88cb-8c52aa69e0a6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:59.877382 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff2357c-2c58-4261-88cb-8c52aa69e0a6 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:59.884178 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 164/333] 10.222.0.18 () {56 vars in 1376 bytes} [Wed Nov 2 07:16:59 2022] GET /identity/v3/role_assignments?group.id=735ae239bcf0421c89bf6c19a919d4ba&scope.project.id=826fa0cd5d134fcab8a9411750769a8f&role.id=b494fedd00f24f0b84a529e8672a156a => generated 611 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:59.896159 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84d8d7b-2f64-4bc7-b6e2-ddc513231e15 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:59.939796 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb672ac2-a290-4b5b-888c-2f540d76dd98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:59.940448 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb672ac2-a290-4b5b-888c-2f540d76dd98 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:59.940725 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb672ac2-a290-4b5b-888c-2f540d76dd98 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:59.940993 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb672ac2-a290-4b5b-888c-2f540d76dd98 None admin] PATH_INFO: `/v3/projects/4ad924d40efc4e06bdd880c155ca36a4` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:59.942207 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb672ac2-a290-4b5b-888c-2f540d76dd98 None admin] RBAC: Authorizing `identity:get_project(project_id=4ad924d40efc4e06bdd880c155ca36a4)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:59.944111 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb672ac2-a290-4b5b-888c-2f540d76dd98 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:59.946051 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 170/334] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Nov 2 07:16:59 2022] GET /identity/v3/projects/4ad924d40efc4e06bdd880c155ca36a4 => generated 344 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:16:59.951896 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff2357c-2c58-4261-88cb-8c52aa69e0a6 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:59.990177 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:00.056970 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff2357c-2c58-4261-88cb-8c52aa69e0a6 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:00.091552 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6faed0f2-6a94-431a-a8d9-1aed50fca363 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:00.092102 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faed0f2-6a94-431a-a8d9-1aed50fca363 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:00.092386 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faed0f2-6a94-431a-a8d9-1aed50fca363 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:00.092661 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faed0f2-6a94-431a-a8d9-1aed50fca363 None admin] PATH_INFO: `/v3/groups/21e85262c94048b4a191fac084f95d4c` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:00.093752 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6faed0f2-6a94-431a-a8d9-1aed50fca363 None admin] RBAC: Authorizing `identity:get_group(group_id=21e85262c94048b4a191fac084f95d4c)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:00.095216 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6faed0f2-6a94-431a-a8d9-1aed50fca363 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:00.096775 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 165/335] 10.222.0.18 () {56 vars in 1180 bytes} [Wed Nov 2 07:16:59 2022] GET /identity/v3/groups/21e85262c94048b4a191fac084f95d4c => generated 229 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:17:00.103933 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb672ac2-a290-4b5b-888c-2f540d76dd98 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:00.203230 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb672ac2-a290-4b5b-888c-2f540d76dd98 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:00.211070 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:00.238857 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d369780e-6a7e-4f3d-8b92-e200a25f6c2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:00.239455 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d369780e-6a7e-4f3d-8b92-e200a25f6c2e None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:00.239731 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d369780e-6a7e-4f3d-8b92-e200a25f6c2e None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:00.239998 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d369780e-6a7e-4f3d-8b92-e200a25f6c2e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:00.240801 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d369780e-6a7e-4f3d-8b92-e200a25f6c2e None admin] RBAC: Adding query filter params (group.id=21e85262c94048b4a191fac084f95d4c, role.id=c4bf019fc6c44c71a6c984f6d0bab75f, scope.project.id=4ad924d40efc4e06bdd880c155ca36a4) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:17:00.241010 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d369780e-6a7e-4f3d-8b92-e200a25f6c2e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:00.242448 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d369780e-6a7e-4f3d-8b92-e200a25f6c2e None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:00.248290 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 171/336] 10.222.0.18 () {56 vars in 1376 bytes} [Wed Nov 2 07:17:00 2022] GET /identity/v3/role_assignments?group.id=21e85262c94048b4a191fac084f95d4c&scope.project.id=4ad924d40efc4e06bdd880c155ca36a4&role.id=c4bf019fc6c44c71a6c984f6d0bab75f => generated 611 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:17:02.793945 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5713a3-4922-444b-963a-0c9579b900d3 None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:02.795014 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5713a3-4922-444b-963a-0c9579b900d3 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:02.795512 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5713a3-4922-444b-963a-0c9579b900d3 None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:02.797399 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 166/337] 10.222.0.18 () {54 vars in 927 bytes} [Wed Nov 2 07:17:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:17:02.803260 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6e5f45-24b4-4b64-8d45-500de788bc5b None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:02.804154 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6e5f45-24b4-4b64-8d45-500de788bc5b None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:02.804626 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6e5f45-24b4-4b64-8d45-500de788bc5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:02.827804 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-de6e5f45-24b4-4b64-8d45-500de788bc5b None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:17:02.829329 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-de6e5f45-24b4-4b64-8d45-500de788bc5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:17:02.914492 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 172/338] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Nov 2 07:17:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:17:03.328890 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7a0dab-d25f-4a6a-a5f3-2f3180d65b86 None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:03.328890 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7a0dab-d25f-4a6a-a5f3-2f3180d65b86 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:03.328890 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7a0dab-d25f-4a6a-a5f3-2f3180d65b86 None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:03.328890 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 167/339] 10.222.0.18 () {54 vars in 927 bytes} [Wed Nov 2 07:17:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:17:03.336034 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7aa0bc2-71ab-406f-9b8a-b1964b2a5f3b None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:03.336569 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7aa0bc2-71ab-406f-9b8a-b1964b2a5f3b None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:03.337042 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7aa0bc2-71ab-406f-9b8a-b1964b2a5f3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:03.358338 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-e7aa0bc2-71ab-406f-9b8a-b1964b2a5f3b None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:17:03.366146 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-e7aa0bc2-71ab-406f-9b8a-b1964b2a5f3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:17:03.367561 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7a0dab-d25f-4a6a-a5f3-2f3180d65b86 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:03.462245 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 173/340] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Nov 2 07:17:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Nov 02 07:17:03.470381 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e228196b-ec74-4f8f-a21f-aea26ef91e05 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:03.470381 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e228196b-ec74-4f8f-a21f-aea26ef91e05 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:03.470381 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e228196b-ec74-4f8f-a21f-aea26ef91e05 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:03.470381 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 174/341] 10.222.0.18 () {54 vars in 927 bytes} [Wed Nov 2 07:17:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:17:03.480731 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cd2723-2f5e-4d8d-b6a5-f9c3b3890ea9 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:03.481011 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cd2723-2f5e-4d8d-b6a5-f9c3b3890ea9 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:03.481286 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cd2723-2f5e-4d8d-b6a5-f9c3b3890ea9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:03.507826 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-53cd2723-2f5e-4d8d-b6a5-f9c3b3890ea9 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:17:03.509016 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-53cd2723-2f5e-4d8d-b6a5-f9c3b3890ea9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:17:03.539538 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7a0dab-d25f-4a6a-a5f3-2f3180d65b86 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:03.542757 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:03.567679 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:03.581255 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0610e481-f014-4dac-a4f0-a79c3000e9df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:03.581822 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0610e481-f014-4dac-a4f0-a79c3000e9df None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:03.582102 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0610e481-f014-4dac-a4f0-a79c3000e9df None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:03.582390 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0610e481-f014-4dac-a4f0-a79c3000e9df None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:03.594516 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0610e481-f014-4dac-a4f0-a79c3000e9df None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:03.596064 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0610e481-f014-4dac-a4f0-a79c3000e9df None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:03.613641 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-0610e481-f014-4dac-a4f0-a79c3000e9df None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:17:03.613641 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:17:03.613641 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:17:03.613641 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:17:03.613641 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:17:03.613641 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:17:03.613641 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:17:03.613641 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:17:03.613641 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:17:03.613641 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:17:03.613641 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:17:03.613641 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:17:03.613641 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:17:03.613641 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:17:03.613641 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:17:03.613641 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:17:03.614716 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:17:03.614716 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:17:03.614716 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:17:03.614716 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:17:03.614716 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:17:03.614716 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:17:03.614716 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:17:03.614716 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Nov 02 07:17:03.614716 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:17:03.614716 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:17:03.614716 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:17:03.614716 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:17:03.614716 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:17:03.614716 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:17:03.614716 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:17:03.614716 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:17:03.614716 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:17:03.615651 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:17:03.615651 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:17:03.615651 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Nov 02 07:17:03.615651 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 168/342] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Nov 2 07:17:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 07:17:03.620855 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0610e481-f014-4dac-a4f0-a79c3000e9df None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:03.697097 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 175/343] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Nov 2 07:17:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Nov 02 07:17:03.698463 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683ce56f-1fcc-4d32-8df2-43880d938b91 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:03.698753 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683ce56f-1fcc-4d32-8df2-43880d938b91 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:03.699028 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683ce56f-1fcc-4d32-8df2-43880d938b91 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:03.700655 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 176/344] 10.222.0.18 () {54 vars in 927 bytes} [Wed Nov 2 07:17:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:17:03.705919 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a2ecfe-e7b2-433e-8cfc-a5fdb0379aa9 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:03.706198 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a2ecfe-e7b2-433e-8cfc-a5fdb0379aa9 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:03.706584 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a2ecfe-e7b2-433e-8cfc-a5fdb0379aa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:03.735112 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0610e481-f014-4dac-a4f0-a79c3000e9df None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:03.737403 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-80a2ecfe-e7b2-433e-8cfc-a5fdb0379aa9 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:17:03.738592 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-80a2ecfe-e7b2-433e-8cfc-a5fdb0379aa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:17:03.794483 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab9cc44-98c3-4475-8615-01719023ed1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:03.795193 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab9cc44-98c3-4475-8615-01719023ed1d None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:03.795477 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab9cc44-98c3-4475-8615-01719023ed1d None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:03.796342 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab9cc44-98c3-4475-8615-01719023ed1d None admin] PATH_INFO: `/v3/domains` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:03.797258 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab9cc44-98c3-4475-8615-01719023ed1d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:17:03.797563 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab9cc44-98c3-4475-8615-01719023ed1d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:03.799579 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab9cc44-98c3-4475-8615-01719023ed1d None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:03.826486 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 169/345] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Nov 2 07:17:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:17:03.827356 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab9cc44-98c3-4475-8615-01719023ed1d None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:03.847800 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:03.855086 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 177/346] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Nov 2 07:17:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:17:03.855987 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a2ecfe-e7b2-433e-8cfc-a5fdb0379aa9 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:03.999101 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:04.004541 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab9cc44-98c3-4475-8615-01719023ed1d None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:04.019031 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a2ecfe-e7b2-433e-8cfc-a5fdb0379aa9 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:04.052497 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6723c758-97c1-4ca0-a1ad-969e5a7bdb51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:04.052985 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6723c758-97c1-4ca0-a1ad-969e5a7bdb51 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:04.053260 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6723c758-97c1-4ca0-a1ad-969e5a7bdb51 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:04.053531 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6723c758-97c1-4ca0-a1ad-969e5a7bdb51 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:04.063798 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6723c758-97c1-4ca0-a1ad-969e5a7bdb51 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:04.064760 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9ed478-cbdd-4823-8f5a-b494ce6e392d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:04.065241 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6723c758-97c1-4ca0-a1ad-969e5a7bdb51 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:04.065837 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9ed478-cbdd-4823-8f5a-b494ce6e392d None admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:04.066212 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9ed478-cbdd-4823-8f5a-b494ce6e392d None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:04.066617 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9ed478-cbdd-4823-8f5a-b494ce6e392d None admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:04.067449 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9ed478-cbdd-4823-8f5a-b494ce6e392d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:04.069596 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9ed478-cbdd-4823-8f5a-b494ce6e392d None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:04.081866 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-6723c758-97c1-4ca0-a1ad-969e5a7bdb51 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:17:04.081866 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:17:04.081866 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:17:04.081866 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:17:04.081866 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:17:04.081866 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:17:04.081866 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:17:04.081866 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:17:04.081866 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:17:04.081866 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:17:04.081866 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:17:04.081866 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:17:04.081866 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:17:04.081866 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:17:04.081866 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:17:04.081866 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:17:04.082901 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:17:04.082901 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:17:04.082901 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:17:04.082901 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:17:04.082901 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:17:04.082901 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:17:04.082901 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:17:04.082901 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Nov 02 07:17:04.082901 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:17:04.082901 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:17:04.082901 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:17:04.082901 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:17:04.082901 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:17:04.082901 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:17:04.082901 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:17:04.082901 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:17:04.082901 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:17:04.084384 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:17:04.084384 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:17:04.084384 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Nov 02 07:17:04.090587 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 170/347] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Nov 2 07:17:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:17:04.091506 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6723c758-97c1-4ca0-a1ad-969e5a7bdb51 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:04.102462 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 178/348] 10.222.0.18 () {60 vars in 1132 bytes} [Wed Nov 2 07:17:03 2022] POST /identity/v3/users => generated 246 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:17:04.102462 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9ed478-cbdd-4823-8f5a-b494ce6e392d None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:04.210570 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:04.214724 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9ed478-cbdd-4823-8f5a-b494ce6e392d None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:04.253501 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebc14d1-9328-4c6c-9194-7df2c3138390 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:04.254117 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebc14d1-9328-4c6c-9194-7df2c3138390 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:04.254413 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebc14d1-9328-4c6c-9194-7df2c3138390 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:04.254687 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebc14d1-9328-4c6c-9194-7df2c3138390 None admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:04.255335 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebc14d1-9328-4c6c-9194-7df2c3138390 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:17:04.255675 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebc14d1-9328-4c6c-9194-7df2c3138390 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:04.257173 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebc14d1-9328-4c6c-9194-7df2c3138390 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:04.260443 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6723c758-97c1-4ca0-a1ad-969e5a7bdb51 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:04.270569 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 179/349] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Nov 2 07:17:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:17:04.271278 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebc14d1-9328-4c6c-9194-7df2c3138390 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:04.280714 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:04.305193 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd84a8e0-309a-45bf-8d64-21a10999a4dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:04.306137 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd84a8e0-309a-45bf-8d64-21a10999a4dd None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:04.306443 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd84a8e0-309a-45bf-8d64-21a10999a4dd None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:04.306990 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd84a8e0-309a-45bf-8d64-21a10999a4dd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:04.319843 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd84a8e0-309a-45bf-8d64-21a10999a4dd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:04.321344 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd84a8e0-309a-45bf-8d64-21a10999a4dd None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:04.338609 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-dd84a8e0-309a-45bf-8d64-21a10999a4dd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:17:04.338609 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:17:04.338609 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:17:04.338609 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:17:04.338609 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:17:04.338609 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:17:04.338609 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:17:04.338609 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:17:04.338609 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:17:04.338609 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:17:04.338609 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:17:04.338609 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:17:04.338609 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:17:04.338609 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:17:04.338609 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:17:04.338609 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:17:04.339789 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:17:04.339789 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:17:04.339789 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:17:04.339789 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:17:04.339789 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:17:04.339789 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:17:04.339789 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:17:04.339789 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Nov 02 07:17:04.339789 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:17:04.339789 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:17:04.339789 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:17:04.339789 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:17:04.339789 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:17:04.339789 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:17:04.339789 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:17:04.339789 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:17:04.339789 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:17:04.340711 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:17:04.340711 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:17:04.340711 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Nov 02 07:17:04.340711 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 171/350] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Nov 2 07:17:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:17:04.340949 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd84a8e0-309a-45bf-8d64-21a10999a4dd None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:04.392441 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebc14d1-9328-4c6c-9194-7df2c3138390 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:04.432352 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfbbad8-b3f0-41ff-9bfd-08c346c75283 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:04.433241 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfbbad8-b3f0-41ff-9bfd-08c346c75283 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:04.433988 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfbbad8-b3f0-41ff-9bfd-08c346c75283 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:04.434267 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfbbad8-b3f0-41ff-9bfd-08c346c75283 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:04.449067 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfbbad8-b3f0-41ff-9bfd-08c346c75283 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:04.451028 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfbbad8-b3f0-41ff-9bfd-08c346c75283 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:04.453373 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd84a8e0-309a-45bf-8d64-21a10999a4dd None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:04.466199 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-abfbbad8-b3f0-41ff-9bfd-08c346c75283 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:17:04.466199 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:17:04.466199 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:17:04.466199 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:17:04.466199 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:17:04.466199 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:17:04.466199 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:17:04.466199 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:17:04.466199 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:17:04.466199 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:17:04.466199 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:17:04.466199 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:17:04.466199 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:17:04.466199 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:17:04.466199 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:17:04.466199 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:17:04.467144 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:17:04.467144 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:17:04.467144 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:17:04.467144 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:17:04.467144 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:17:04.467144 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:17:04.467144 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:17:04.467144 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:17:04.467144 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:17:04.467144 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:17:04.467144 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:17:04.467144 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:17:04.467144 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:17:04.467144 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:17:04.467144 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:17:04.467144 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:17:04.467144 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:17:04.467969 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:17:04.467969 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:17:04.467969 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:17:04.467969 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 180/351] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Nov 2 07:17:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:17:04.468736 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292b3f92-bae6-4eca-ab3d-4df6257f092e None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:04.468844 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292b3f92-bae6-4eca-ab3d-4df6257f092e None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:04.469074 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292b3f92-bae6-4eca-ab3d-4df6257f092e None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:04.470486 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 181/352] 10.222.0.18 () {54 vars in 927 bytes} [Wed Nov 2 07:17:04 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:17:04.475223 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292b3f92-bae6-4eca-ab3d-4df6257f092e None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:04.482781 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:04.500378 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96316a55-2892-4e50-a794-47b042112016 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:04.501002 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96316a55-2892-4e50-a794-47b042112016 None admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:04.501280 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96316a55-2892-4e50-a794-47b042112016 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:04.501558 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96316a55-2892-4e50-a794-47b042112016 None admin] PATH_INFO: `/v3/users` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:04.502206 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96316a55-2892-4e50-a794-47b042112016 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:04.503729 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96316a55-2892-4e50-a794-47b042112016 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:04.527840 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 172/353] 10.222.0.18 () {60 vars in 1132 bytes} [Wed Nov 2 07:17:04 2022] POST /identity/v3/users => generated 248 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:17:04.528660 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96316a55-2892-4e50-a794-47b042112016 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:04.548182 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:04.602505 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292b3f92-bae6-4eca-ab3d-4df6257f092e None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:04.641459 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600058f0-dba2-4763-8a95-bf5ca478d9ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:04.642663 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600058f0-dba2-4763-8a95-bf5ca478d9ce None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:04.643230 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600058f0-dba2-4763-8a95-bf5ca478d9ce None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:04.643690 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600058f0-dba2-4763-8a95-bf5ca478d9ce None admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:04.644552 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600058f0-dba2-4763-8a95-bf5ca478d9ce None admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:17:04.645048 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600058f0-dba2-4763-8a95-bf5ca478d9ce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:04.647397 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600058f0-dba2-4763-8a95-bf5ca478d9ce None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:04.654335 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96316a55-2892-4e50-a794-47b042112016 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:04.667265 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 182/354] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Nov 2 07:17:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:17:04.669032 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ee82dc-7527-460c-817b-24ba15bbec84 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:04.669538 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ee82dc-7527-460c-817b-24ba15bbec84 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:04.670079 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ee82dc-7527-460c-817b-24ba15bbec84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:04.691951 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-72ee82dc-7527-460c-817b-24ba15bbec84 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:17:04.693811 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-72ee82dc-7527-460c-817b-24ba15bbec84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:17:04.695975 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6347157e-3f9a-4714-a9e4-ba73e3ef6c83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:04.696686 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6347157e-3f9a-4714-a9e4-ba73e3ef6c83 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:04.697023 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6347157e-3f9a-4714-a9e4-ba73e3ef6c83 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:04.697404 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6347157e-3f9a-4714-a9e4-ba73e3ef6c83 None admin] PATH_INFO: `/v3/domains` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:04.697949 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6347157e-3f9a-4714-a9e4-ba73e3ef6c83 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:17:04.698253 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6347157e-3f9a-4714-a9e4-ba73e3ef6c83 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:04.699746 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6347157e-3f9a-4714-a9e4-ba73e3ef6c83 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:04.719778 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 173/355] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Nov 2 07:17:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:17:04.720729 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6347157e-3f9a-4714-a9e4-ba73e3ef6c83 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:04.765475 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:04.779378 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 183/356] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Nov 2 07:17:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:17:04.779378 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ee82dc-7527-460c-817b-24ba15bbec84 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:04.834284 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6347157e-3f9a-4714-a9e4-ba73e3ef6c83 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:04.846759 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:04.873773 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c410376f-975c-438d-b1a3-9b9cf19b0b9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:04.874788 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c410376f-975c-438d-b1a3-9b9cf19b0b9b None admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:04.875267 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c410376f-975c-438d-b1a3-9b9cf19b0b9b None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:04.875743 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c410376f-975c-438d-b1a3-9b9cf19b0b9b None admin] PATH_INFO: `/v3/users` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:04.876636 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c410376f-975c-438d-b1a3-9b9cf19b0b9b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:04.878291 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c410376f-975c-438d-b1a3-9b9cf19b0b9b None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:04.917480 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ee82dc-7527-460c-817b-24ba15bbec84 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:04.921720 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 174/357] 10.222.0.18 () {60 vars in 1132 bytes} [Wed Nov 2 07:17:04 2022] POST /identity/v3/users => generated 248 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:17:04.976209 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cce69b-5d80-418a-955a-e3241f6c03d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:04.977275 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cce69b-5d80-418a-955a-e3241f6c03d2 None admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:04.977745 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cce69b-5d80-418a-955a-e3241f6c03d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:04.978514 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cce69b-5d80-418a-955a-e3241f6c03d2 None admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:04.979365 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cce69b-5d80-418a-955a-e3241f6c03d2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:04.981000 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cce69b-5d80-418a-955a-e3241f6c03d2 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:05.027614 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 184/358] 10.222.0.18 () {60 vars in 1132 bytes} [Wed Nov 2 07:17:04 2022] POST /identity/v3/users => generated 249 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:17:05.215012 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c410376f-975c-438d-b1a3-9b9cf19b0b9b None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:05.320328 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:05.329286 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c410376f-975c-438d-b1a3-9b9cf19b0b9b None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:05.367684 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4453e01a-9e33-41bf-8a2b-68914a0effd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:05.368311 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4453e01a-9e33-41bf-8a2b-68914a0effd0 None admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:05.368593 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4453e01a-9e33-41bf-8a2b-68914a0effd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:05.368862 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4453e01a-9e33-41bf-8a2b-68914a0effd0 None admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:05.369536 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4453e01a-9e33-41bf-8a2b-68914a0effd0 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:05.370966 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4453e01a-9e33-41bf-8a2b-68914a0effd0 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:05.379675 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.sql.core [None req-4453e01a-9e33-41bf-8a2b-68914a0effd0 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Nov 02 07:17:05.379675 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Nov 02 07:17:05.379675 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [parameters: {'id': '749b68641ac24bf589eb651fea53325f', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Nov 02 07:17:05.379675 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81501) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Nov 02 07:17:05.399038 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-4453e01a-9e33-41bf-8a2b-68914a0effd0 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Nov 02 07:17:05.399038 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:17:05.399038 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Nov 02 07:17:05.399038 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application self.dialect.do_execute( Nov 02 07:17:05.399038 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Nov 02 07:17:05.399038 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Nov 02 07:17:05.399038 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Nov 02 07:17:05.399038 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application result = self._query(query) Nov 02 07:17:05.399038 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Nov 02 07:17:05.399038 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application conn.query(q) Nov 02 07:17:05.399038 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Nov 02 07:17:05.399038 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Nov 02 07:17:05.399038 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Nov 02 07:17:05.399038 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application result.read() Nov 02 07:17:05.399038 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Nov 02 07:17:05.400106 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Nov 02 07:17:05.400106 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Nov 02 07:17:05.400106 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application packet.raise_for_error() Nov 02 07:17:05.400106 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Nov 02 07:17:05.400106 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Nov 02 07:17:05.400106 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Nov 02 07:17:05.400106 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Nov 02 07:17:05.400106 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Nov 02 07:17:05.400106 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Nov 02 07:17:05.400106 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Nov 02 07:17:05.400106 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Nov 02 07:17:05.400106 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:17:05.400106 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Nov 02 07:17:05.400106 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return method(*args, **kwargs) Nov 02 07:17:05.400106 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Nov 02 07:17:05.400106 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return ref.to_dict() Nov 02 07:17:05.400106 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Nov 02 07:17:05.400106 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application next(self.gen) Nov 02 07:17:05.401025 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1068, in _transaction_scope Nov 02 07:17:05.401025 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application yield resource Nov 02 07:17:05.401025 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Nov 02 07:17:05.401025 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application next(self.gen) Nov 02 07:17:05.401025 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 670, in _session Nov 02 07:17:05.401025 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application self.session.rollback() Nov 02 07:17:05.401025 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Nov 02 07:17:05.401025 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application self.force_reraise() Nov 02 07:17:05.401025 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Nov 02 07:17:05.401025 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise self.value Nov 02 07:17:05.401025 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 667, in _session Nov 02 07:17:05.401025 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Nov 02 07:17:05.401025 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 695, in _end_session_transaction Nov 02 07:17:05.401025 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application session.commit() Nov 02 07:17:05.401025 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Nov 02 07:17:05.401025 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Nov 02 07:17:05.401025 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Nov 02 07:17:05.401960 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application self._prepare_impl() Nov 02 07:17:05.401960 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Nov 02 07:17:05.401960 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application self.session.flush() Nov 02 07:17:05.401960 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Nov 02 07:17:05.401960 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application self._flush(objects) Nov 02 07:17:05.401960 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Nov 02 07:17:05.401960 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Nov 02 07:17:05.401960 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Nov 02 07:17:05.401960 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application compat.raise_( Nov 02 07:17:05.401960 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Nov 02 07:17:05.401960 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception Nov 02 07:17:05.401960 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Nov 02 07:17:05.401960 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application flush_context.execute() Nov 02 07:17:05.401960 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Nov 02 07:17:05.401960 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rec.execute(self) Nov 02 07:17:05.401960 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Nov 02 07:17:05.401960 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Nov 02 07:17:05.402855 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Nov 02 07:17:05.402855 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application _emit_insert_statements( Nov 02 07:17:05.402855 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Nov 02 07:17:05.402855 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application c = connection._execute_20( Nov 02 07:17:05.402855 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Nov 02 07:17:05.402855 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Nov 02 07:17:05.402855 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Nov 02 07:17:05.402855 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Nov 02 07:17:05.402855 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Nov 02 07:17:05.402855 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application ret = self._execute_context( Nov 02 07:17:05.402855 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Nov 02 07:17:05.402855 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Nov 02 07:17:05.402855 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Nov 02 07:17:05.402855 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Nov 02 07:17:05.402855 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Nov 02 07:17:05.402855 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception Nov 02 07:17:05.403635 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Nov 02 07:17:05.403635 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application self.dialect.do_execute( Nov 02 07:17:05.403635 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Nov 02 07:17:05.403635 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Nov 02 07:17:05.403635 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Nov 02 07:17:05.403635 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application result = self._query(query) Nov 02 07:17:05.403635 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Nov 02 07:17:05.403635 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application conn.query(q) Nov 02 07:17:05.403635 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Nov 02 07:17:05.403635 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Nov 02 07:17:05.403635 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Nov 02 07:17:05.403635 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application result.read() Nov 02 07:17:05.403635 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Nov 02 07:17:05.403635 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Nov 02 07:17:05.403635 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Nov 02 07:17:05.403635 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application packet.raise_for_error() Nov 02 07:17:05.404396 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Nov 02 07:17:05.404396 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Nov 02 07:17:05.404396 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Nov 02 07:17:05.404396 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Nov 02 07:17:05.404396 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Nov 02 07:17:05.404396 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Nov 02 07:17:05.404396 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [parameters: {'id': '749b68641ac24bf589eb651fea53325f', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Nov 02 07:17:05.404396 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Nov 02 07:17:05.404396 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Nov 02 07:17:05.404396 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 07:17:05.404396 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Nov 02 07:17:05.404396 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:17:05.404396 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:17:05.404396 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:17:05.404396 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:17:05.404396 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:17:05.405261 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:17:05.405261 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:17:05.405261 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:17:05.405261 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:17:05.405261 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:17:05.405261 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:17:05.405261 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Nov 02 07:17:05.405261 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Nov 02 07:17:05.405261 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:17:05.405261 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:17:05.405261 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Nov 02 07:17:05.405261 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Nov 02 07:17:05.405261 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Nov 02 07:17:05.405261 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Nov 02 07:17:05.405261 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Nov 02 07:17:05.405261 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Nov 02 07:17:05.406130 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 175/359] 10.222.0.18 () {60 vars in 1131 bytes} [Wed Nov 2 07:17:05 2022] POST /identity/v3/roles => generated 144 bytes in 186 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:17:05.410540 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cce69b-5d80-418a-955a-e3241f6c03d2 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:05.527249 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cce69b-5d80-418a-955a-e3241f6c03d2 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:05.529574 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:05.575172 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b88033-e507-44fc-82ac-ad6261055de9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:05.575823 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b88033-e507-44fc-82ac-ad6261055de9 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:05.576105 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b88033-e507-44fc-82ac-ad6261055de9 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:05.576378 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b88033-e507-44fc-82ac-ad6261055de9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:05.594662 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b88033-e507-44fc-82ac-ad6261055de9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:17:05.601240 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-67b88033-e507-44fc-82ac-ad6261055de9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Nov 02 07:17:05.601240 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:17:05.601240 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:17:05.601240 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:17:05.601240 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:17:05.601240 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:17:05.601240 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:17:05.601240 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:17:05.601240 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:17:05.601240 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:17:05.601240 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:17:05.601240 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:17:05.601240 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:17:05.601240 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:17:05.601240 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:17:05.601240 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:17:05.602222 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:17:05.602222 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:17:05.602222 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:17:05.602222 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:17:05.602222 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:17:05.602222 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:17:05.602222 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:17:05.602222 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:17:05.602222 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:17:05.602222 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:17:05.602222 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:17:05.602222 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:17:05.602222 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:17:05.602222 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:17:05.602222 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:17:05.602222 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:17:05.603340 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:17:05.603340 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:17:05.603340 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:17:05.603340 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:17:05.603340 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:17:05.603340 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:17:05.603340 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:17:05.603340 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:17:05.603340 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:17:05.603340 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:17:05.603340 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Nov 02 07:17:05.603340 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:17:05.607819 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 185/360] 10.222.0.18 () {56 vars in 1114 bytes} [Wed Nov 2 07:17:05 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:17:05.609221 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4453e01a-9e33-41bf-8a2b-68914a0effd0 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:05.727054 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4453e01a-9e33-41bf-8a2b-68914a0effd0 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:05.763293 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c477bc-cfe8-40a2-bca3-9b161f45b0e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:05.763904 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c477bc-cfe8-40a2-bca3-9b161f45b0e9 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:05.764247 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c477bc-cfe8-40a2-bca3-9b161f45b0e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:05.764524 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c477bc-cfe8-40a2-bca3-9b161f45b0e9 None admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:05.765187 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c477bc-cfe8-40a2-bca3-9b161f45b0e9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:17:05.765490 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c477bc-cfe8-40a2-bca3-9b161f45b0e9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:05.766914 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c477bc-cfe8-40a2-bca3-9b161f45b0e9 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:05.780838 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 176/361] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Nov 2 07:17:05 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:17:06.535518 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66f608d-1d0f-4aa7-bb64-bbc51f6b5d09 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:06.536200 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66f608d-1d0f-4aa7-bb64-bbc51f6b5d09 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:06.536200 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66f608d-1d0f-4aa7-bb64-bbc51f6b5d09 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:06.537987 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 186/362] 10.222.0.18 () {54 vars in 927 bytes} [Wed Nov 2 07:17:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:17:06.543302 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb619d8c-3abf-4626-a1c4-7791bfbb6bf0 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:06.543810 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb619d8c-3abf-4626-a1c4-7791bfbb6bf0 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:06.543962 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb619d8c-3abf-4626-a1c4-7791bfbb6bf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:06.566974 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-eb619d8c-3abf-4626-a1c4-7791bfbb6bf0 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:17:06.569052 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-eb619d8c-3abf-4626-a1c4-7791bfbb6bf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:17:06.583758 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:06.644271 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 177/363] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Nov 2 07:17:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:17:07.088418 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66f608d-1d0f-4aa7-bb64-bbc51f6b5d09 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:07.145027 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e710be5e-0b9b-4103-a292-b85315f34db7 None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:07.145679 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e710be5e-0b9b-4103-a292-b85315f34db7 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:07.146281 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e710be5e-0b9b-4103-a292-b85315f34db7 None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:07.148129 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 178/364] 10.222.0.18 () {54 vars in 927 bytes} [Wed Nov 2 07:17:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:17:07.153605 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda72735-5e40-4c47-b138-7cbf588e0cef None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:07.154170 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda72735-5e40-4c47-b138-7cbf588e0cef None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:07.154669 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda72735-5e40-4c47-b138-7cbf588e0cef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:07.187754 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-eda72735-5e40-4c47-b138-7cbf588e0cef None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:17:07.189963 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-eda72735-5e40-4c47-b138-7cbf588e0cef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:17:07.256569 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66f608d-1d0f-4aa7-bb64-bbc51f6b5d09 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:07.324636 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3535754-a414-4c03-8d8f-41ad33408115 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:07.325280 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3535754-a414-4c03-8d8f-41ad33408115 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:07.325557 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3535754-a414-4c03-8d8f-41ad33408115 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:07.325831 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3535754-a414-4c03-8d8f-41ad33408115 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:07.328587 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 179/365] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Nov 2 07:17:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Nov 02 07:17:07.350718 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3535754-a414-4c03-8d8f-41ad33408115 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:17:07.355959 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-c3535754-a414-4c03-8d8f-41ad33408115 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Nov 02 07:17:07.355959 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:17:07.355959 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:17:07.355959 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:17:07.355959 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:17:07.355959 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:17:07.355959 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:17:07.355959 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:17:07.355959 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:17:07.355959 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:17:07.355959 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:17:07.355959 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:17:07.355959 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:17:07.355959 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:17:07.355959 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:17:07.355959 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:17:07.356917 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:17:07.356917 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:17:07.356917 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:17:07.356917 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:17:07.356917 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:17:07.356917 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:17:07.356917 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:17:07.356917 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:17:07.356917 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:17:07.356917 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:17:07.356917 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:17:07.356917 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:17:07.356917 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:17:07.356917 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:17:07.356917 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:17:07.356917 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:17:07.357696 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:17:07.357696 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:17:07.357696 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:17:07.357696 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:17:07.357696 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:17:07.357696 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:17:07.357696 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:17:07.357696 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:17:07.357696 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:17:07.357696 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:17:07.357696 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Nov 02 07:17:07.357696 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:17:07.357696 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 187/366] 10.222.0.18 () {56 vars in 1102 bytes} [Wed Nov 2 07:17:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:17:07.365328 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda72735-5e40-4c47-b138-7cbf588e0cef None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:07.482740 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:07.505033 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda72735-5e40-4c47-b138-7cbf588e0cef None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:07.554101 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d46d00-3baa-4e83-aff3-8c67f7feb575 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:07.554894 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d46d00-3baa-4e83-aff3-8c67f7feb575 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:07.555076 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d46d00-3baa-4e83-aff3-8c67f7feb575 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:07.555348 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d46d00-3baa-4e83-aff3-8c67f7feb575 None admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:07.556017 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d46d00-3baa-4e83-aff3-8c67f7feb575 None admin] RBAC: Adding query filter params (name=service) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:17:07.556313 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d46d00-3baa-4e83-aff3-8c67f7feb575 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:07.557693 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d46d00-3baa-4e83-aff3-8c67f7feb575 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:07.570527 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 180/367] 10.222.0.18 () {56 vars in 1103 bytes} [Wed Nov 2 07:17:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:17:07.571774 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3535754-a414-4c03-8d8f-41ad33408115 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:07.606189 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:07.639610 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d46d00-3baa-4e83-aff3-8c67f7feb575 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:07.742729 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3535754-a414-4c03-8d8f-41ad33408115 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:07.763572 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d46d00-3baa-4e83-aff3-8c67f7feb575 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:07.802014 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d4a66b-cdf7-4ec1-ba1b-560b46075511 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:07.803700 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d4a66b-cdf7-4ec1-ba1b-560b46075511 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:07.804248 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d4a66b-cdf7-4ec1-ba1b-560b46075511 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:07.804889 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d4a66b-cdf7-4ec1-ba1b-560b46075511 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:07.814052 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:07.819002 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d4a66b-cdf7-4ec1-ba1b-560b46075511 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:17:07.820423 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b00f40-1418-486f-8395-cacfb7a78de0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:07.821078 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b00f40-1418-486f-8395-cacfb7a78de0 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:07.821361 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b00f40-1418-486f-8395-cacfb7a78de0 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:07.821634 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b00f40-1418-486f-8395-cacfb7a78de0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:07.826125 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-45d4a66b-cdf7-4ec1-ba1b-560b46075511 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Nov 02 07:17:07.826125 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:17:07.826125 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:17:07.826125 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:17:07.826125 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:17:07.826125 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:17:07.826125 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:17:07.826125 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:17:07.826125 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:17:07.826125 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:17:07.826125 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:17:07.826125 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:17:07.826125 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:17:07.826125 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:17:07.826125 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:17:07.826125 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:17:07.827090 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:17:07.827090 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:17:07.827090 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:17:07.827090 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:17:07.827090 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:17:07.827090 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:17:07.827090 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:17:07.827090 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:17:07.827090 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:17:07.827090 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:17:07.827090 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:17:07.827090 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Nov 02 07:17:07.827090 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:17:07.827090 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:17:07.827090 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:17:07.827090 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:17:07.827871 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:17:07.827871 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:17:07.827871 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:17:07.827871 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:17:07.827871 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:17:07.827871 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:17:07.827871 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:17:07.827871 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:17:07.827871 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:17:07.827871 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:17:07.827871 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Nov 02 07:17:07.827871 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Nov 02 07:17:07.834497 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 181/368] 10.222.0.18 () {56 vars in 1102 bytes} [Wed Nov 2 07:17:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 07:17:07.837132 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d4a66b-cdf7-4ec1-ba1b-560b46075511 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:07.848942 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b00f40-1418-486f-8395-cacfb7a78de0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:07.851320 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b00f40-1418-486f-8395-cacfb7a78de0 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:07.879939 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-10b00f40-1418-486f-8395-cacfb7a78de0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:17:07.879939 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:17:07.879939 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:17:07.879939 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:17:07.879939 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:17:07.879939 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:17:07.879939 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:17:07.879939 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:17:07.879939 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:17:07.879939 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:17:07.879939 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:17:07.879939 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:17:07.879939 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:17:07.879939 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:17:07.879939 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:17:07.879939 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:17:07.881425 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:17:07.881425 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:17:07.881425 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:17:07.881425 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:17:07.881425 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:17:07.881425 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:17:07.881425 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:17:07.881425 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:17:07.881425 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:17:07.881425 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:17:07.881425 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:17:07.881425 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:17:07.881425 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:17:07.881425 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:17:07.881425 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:17:07.881425 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:17:07.881425 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:17:07.882724 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:17:07.882724 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:17:07.882724 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:17:07.882724 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 188/369] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Nov 2 07:17:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:17:07.883133 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dba264f-157d-4ce0-a6b5-4cc057dec0fc None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:07.883332 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dba264f-157d-4ce0-a6b5-4cc057dec0fc None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:07.883602 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dba264f-157d-4ce0-a6b5-4cc057dec0fc None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:07.884811 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 189/370] 10.222.0.18 () {54 vars in 927 bytes} [Wed Nov 2 07:17:07 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:17:07.889469 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dba264f-157d-4ce0-a6b5-4cc057dec0fc None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:07.952483 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d4a66b-cdf7-4ec1-ba1b-560b46075511 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:07.970711 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:07.987826 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccea1ca-e819-440d-b2c4-09a26ecb45a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:07.988434 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccea1ca-e819-440d-b2c4-09a26ecb45a7 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:07.988716 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccea1ca-e819-440d-b2c4-09a26ecb45a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:07.988993 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccea1ca-e819-440d-b2c4-09a26ecb45a7 None admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:07.989650 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccea1ca-e819-440d-b2c4-09a26ecb45a7 None admin] RBAC: Adding query filter params (name=service) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:17:07.990621 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccea1ca-e819-440d-b2c4-09a26ecb45a7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:07.992098 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccea1ca-e819-440d-b2c4-09a26ecb45a7 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:07.998343 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dba264f-157d-4ce0-a6b5-4cc057dec0fc None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:08.000212 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 182/371] 10.222.0.18 () {56 vars in 1103 bytes} [Wed Nov 2 07:17:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:17:08.001187 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcf6f4b-75c1-4925-87e4-d301e59dc03e None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:08.001474 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcf6f4b-75c1-4925-87e4-d301e59dc03e None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:08.001748 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcf6f4b-75c1-4925-87e4-d301e59dc03e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:08.020101 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-7dcf6f4b-75c1-4925-87e4-d301e59dc03e None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:17:08.021463 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-7dcf6f4b-75c1-4925-87e4-d301e59dc03e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:17:08.040174 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e093bd-dc85-4542-bfdf-4ed765c037ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:08.040787 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e093bd-dc85-4542-bfdf-4ed765c037ea None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:08.041078 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e093bd-dc85-4542-bfdf-4ed765c037ea None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:08.041338 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e093bd-dc85-4542-bfdf-4ed765c037ea None admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:08.041983 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e093bd-dc85-4542-bfdf-4ed765c037ea None admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:17:08.042387 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e093bd-dc85-4542-bfdf-4ed765c037ea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:08.043740 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e093bd-dc85-4542-bfdf-4ed765c037ea None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:08.057850 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 190/372] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Nov 2 07:17:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:17:08.059315 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fb3e3f-7d96-4ba1-93f3-cf09b5eef1fe None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:08.059596 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fb3e3f-7d96-4ba1-93f3-cf09b5eef1fe None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:08.059865 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fb3e3f-7d96-4ba1-93f3-cf09b5eef1fe None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:08.061182 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 191/373] 10.222.0.18 () {54 vars in 927 bytes} [Wed Nov 2 07:17:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:17:08.061911 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fb3e3f-7d96-4ba1-93f3-cf09b5eef1fe None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:08.090217 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:08.101357 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 183/374] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Nov 2 07:17:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:17:08.102087 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcf6f4b-75c1-4925-87e4-d301e59dc03e None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:08.192144 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fb3e3f-7d96-4ba1-93f3-cf09b5eef1fe None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:08.218494 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcf6f4b-75c1-4925-87e4-d301e59dc03e None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:08.247052 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2eaad0-bca0-4255-aa8d-1e01b7cce379 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:08.248324 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2eaad0-bca0-4255-aa8d-1e01b7cce379 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:08.249112 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2eaad0-bca0-4255-aa8d-1e01b7cce379 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:08.249992 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2eaad0-bca0-4255-aa8d-1e01b7cce379 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:08.254212 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:08.258592 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c9cf8b-2082-49c6-9c56-55b2b7b73e73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:08.259425 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c9cf8b-2082-49c6-9c56-55b2b7b73e73 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:08.259706 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c9cf8b-2082-49c6-9c56-55b2b7b73e73 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:08.259979 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c9cf8b-2082-49c6-9c56-55b2b7b73e73 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:08.267041 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2eaad0-bca0-4255-aa8d-1e01b7cce379 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:08.267567 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c9cf8b-2082-49c6-9c56-55b2b7b73e73 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:08.269029 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2eaad0-bca0-4255-aa8d-1e01b7cce379 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:08.269137 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c9cf8b-2082-49c6-9c56-55b2b7b73e73 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:08.284206 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-c4c9cf8b-2082-49c6-9c56-55b2b7b73e73 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Nov 02 07:17:08.284206 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:17:08.284206 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:17:08.284206 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:17:08.284206 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:17:08.284206 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:17:08.284206 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:17:08.284206 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:17:08.284206 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:17:08.284206 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:17:08.284206 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:17:08.284206 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:17:08.284206 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Nov 02 07:17:08.284206 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_user(user_id) Nov 02 07:17:08.284206 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Nov 02 07:17:08.284206 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Nov 02 07:17:08.285271 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:17:08.285271 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:17:08.285271 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Nov 02 07:17:08.285271 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:17:08.285271 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Nov 02 07:17:08.285271 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:17:08.285271 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:17:08.285271 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:17:08.285271 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:17:08.285271 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:17:08.285271 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:17:08.285271 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Nov 02 07:17:08.285271 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:17:08.285271 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:17:08.285271 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:17:08.285271 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:17:08.285271 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:17:08.285271 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:17:08.291134 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:17:08.291134 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:17:08.291134 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Nov 02 07:17:08.291134 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Nov 02 07:17:08.291134 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Nov 02 07:17:08.291134 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Nov 02 07:17:08.291134 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Nov 02 07:17:08.291134 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Nov 02 07:17:08.291134 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Nov 02 07:17:08.291134 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Nov 02 07:17:08.291134 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 184/375] 10.222.0.18 () {56 vars in 1093 bytes} [Wed Nov 2 07:17:08 2022] GET /identity/v3/users/nova => generated 82 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:17:08.291134 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf3dad1-151b-4fcc-964c-e9b417bf5f8b None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:08.291134 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf3dad1-151b-4fcc-964c-e9b417bf5f8b None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:08.291835 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf3dad1-151b-4fcc-964c-e9b417bf5f8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:08.295744 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-7d2eaad0-bca0-4255-aa8d-1e01b7cce379 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:17:08.295744 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:17:08.295744 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:17:08.295744 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:17:08.295744 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:17:08.295744 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:17:08.295744 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:17:08.295744 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:17:08.295744 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:17:08.295744 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:17:08.295744 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:17:08.295744 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:17:08.295744 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:17:08.295744 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:17:08.295744 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:17:08.295744 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:17:08.296636 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:17:08.296636 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:17:08.296636 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:17:08.296636 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:17:08.296636 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:17:08.296636 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:17:08.296636 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:17:08.296636 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:17:08.296636 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:17:08.296636 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:17:08.296636 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:17:08.296636 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:17:08.296636 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:17:08.296636 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:17:08.296636 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:17:08.296636 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:17:08.296636 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:17:08.297472 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:17:08.297472 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:17:08.297472 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:17:08.299878 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 192/376] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Nov 2 07:17:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:17:08.305216 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2eaad0-bca0-4255-aa8d-1e01b7cce379 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:08.320393 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-5bf3dad1-151b-4fcc-964c-e9b417bf5f8b None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:17:08.322445 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-5bf3dad1-151b-4fcc-964c-e9b417bf5f8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:17:08.385738 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:08.407523 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 185/377] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Nov 2 07:17:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:17:08.408572 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf3dad1-151b-4fcc-964c-e9b417bf5f8b None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:08.418294 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2eaad0-bca0-4255-aa8d-1e01b7cce379 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:08.462984 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77801c1-e3d1-4964-a26d-8bfdd78fd097 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:08.463611 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77801c1-e3d1-4964-a26d-8bfdd78fd097 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:08.464143 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77801c1-e3d1-4964-a26d-8bfdd78fd097 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:08.464561 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77801c1-e3d1-4964-a26d-8bfdd78fd097 None admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:08.465369 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77801c1-e3d1-4964-a26d-8bfdd78fd097 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:17:08.466003 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77801c1-e3d1-4964-a26d-8bfdd78fd097 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:08.468247 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77801c1-e3d1-4964-a26d-8bfdd78fd097 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:08.515587 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf3dad1-151b-4fcc-964c-e9b417bf5f8b None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:08.527736 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 193/378] 10.222.0.18 () {56 vars in 1133 bytes} [Wed Nov 2 07:17:08 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:17:08.528969 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77801c1-e3d1-4964-a26d-8bfdd78fd097 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:08.556957 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5d1937-27a7-4c22-b8a0-501e5e9fe7a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:08.558243 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5d1937-27a7-4c22-b8a0-501e5e9fe7a7 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:08.558742 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5d1937-27a7-4c22-b8a0-501e5e9fe7a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:08.559179 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5d1937-27a7-4c22-b8a0-501e5e9fe7a7 None admin] PATH_INFO: `/v3/domains` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:08.559978 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5d1937-27a7-4c22-b8a0-501e5e9fe7a7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:17:08.560430 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5d1937-27a7-4c22-b8a0-501e5e9fe7a7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:08.562055 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5d1937-27a7-4c22-b8a0-501e5e9fe7a7 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:08.575651 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 186/379] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Nov 2 07:17:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:17:08.581060 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5d1937-27a7-4c22-b8a0-501e5e9fe7a7 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:08.585756 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:08.628014 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:08.659541 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77801c1-e3d1-4964-a26d-8bfdd78fd097 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:08.688721 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5d1937-27a7-4c22-b8a0-501e5e9fe7a7 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:08.699529 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d74bd3-c8c4-486a-8521-76dd83da1d0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:08.700155 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d74bd3-c8c4-486a-8521-76dd83da1d0f None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:08.700431 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d74bd3-c8c4-486a-8521-76dd83da1d0f None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:08.700699 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d74bd3-c8c4-486a-8521-76dd83da1d0f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:08.711244 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d74bd3-c8c4-486a-8521-76dd83da1d0f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:17:08.716094 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-a4d74bd3-c8c4-486a-8521-76dd83da1d0f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Nov 02 07:17:08.716094 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:17:08.716094 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:17:08.716094 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:17:08.716094 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:17:08.716094 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:17:08.716094 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:17:08.716094 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:17:08.716094 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:17:08.716094 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:17:08.716094 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:17:08.716094 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:17:08.716094 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:17:08.716094 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:17:08.716094 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:17:08.716094 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:17:08.717140 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:17:08.717140 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:17:08.717140 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:17:08.717140 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:17:08.717140 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:17:08.717140 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:17:08.717140 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:17:08.717140 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:17:08.717140 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:17:08.717140 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:17:08.717140 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:17:08.717140 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:17:08.717140 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:17:08.717140 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:17:08.717140 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:17:08.717140 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:17:08.718118 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:17:08.718118 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:17:08.718118 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:17:08.718118 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:17:08.718118 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:17:08.718118 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:17:08.718118 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:17:08.718118 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:17:08.718118 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:17:08.718118 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:17:08.718118 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Nov 02 07:17:08.718118 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:17:08.718118 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 194/380] 10.222.0.18 () {56 vars in 1102 bytes} [Wed Nov 2 07:17:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:17:08.718118 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d74bd3-c8c4-486a-8521-76dd83da1d0f None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:08.724577 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0701eb39-8762-4d22-a100-b2c3347011ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:08.725356 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0701eb39-8762-4d22-a100-b2c3347011ad None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:08.725640 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0701eb39-8762-4d22-a100-b2c3347011ad None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:08.725915 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0701eb39-8762-4d22-a100-b2c3347011ad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:08.736796 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0701eb39-8762-4d22-a100-b2c3347011ad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:08.738339 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0701eb39-8762-4d22-a100-b2c3347011ad None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:08.752190 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-0701eb39-8762-4d22-a100-b2c3347011ad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:17:08.752190 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:17:08.752190 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:17:08.752190 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:17:08.752190 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:17:08.752190 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:17:08.752190 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:17:08.752190 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:17:08.752190 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:17:08.752190 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:17:08.752190 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:17:08.752190 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:17:08.752190 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:17:08.752190 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:17:08.752190 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:17:08.752190 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:17:08.753269 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:17:08.753269 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:17:08.753269 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:17:08.753269 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:17:08.753269 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:17:08.753269 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:17:08.753269 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:17:08.753269 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Nov 02 07:17:08.753269 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:17:08.753269 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:17:08.753269 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:17:08.753269 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:17:08.753269 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:17:08.753269 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:17:08.753269 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:17:08.753269 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:17:08.753269 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:17:08.754299 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:17:08.754299 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:17:08.754299 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Nov 02 07:17:08.754299 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 187/381] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Nov 2 07:17:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:17:08.754691 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9060ac2c-913d-433c-9787-e8155bce2157 None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:08.754969 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9060ac2c-913d-433c-9787-e8155bce2157 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:08.755235 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9060ac2c-913d-433c-9787-e8155bce2157 None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:08.756964 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 188/382] 10.222.0.18 () {54 vars in 927 bytes} [Wed Nov 2 07:17:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:17:08.757632 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9060ac2c-913d-433c-9787-e8155bce2157 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:08.821568 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d74bd3-c8c4-486a-8521-76dd83da1d0f None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:08.828323 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:08.849325 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:08.857541 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3595108-85ea-445a-9038-758426a6d4b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:08.858175 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3595108-85ea-445a-9038-758426a6d4b4 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:08.858280 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9060ac2c-913d-433c-9787-e8155bce2157 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:08.858486 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3595108-85ea-445a-9038-758426a6d4b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:08.858759 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3595108-85ea-445a-9038-758426a6d4b4 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:08.866485 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3595108-85ea-445a-9038-758426a6d4b4 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:08.867936 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3595108-85ea-445a-9038-758426a6d4b4 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:08.880143 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-c3595108-85ea-445a-9038-758426a6d4b4 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Nov 02 07:17:08.880143 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:17:08.880143 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:17:08.880143 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:17:08.880143 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:17:08.880143 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:17:08.880143 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:17:08.880143 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:17:08.880143 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:17:08.880143 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:17:08.880143 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:17:08.880143 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:17:08.880143 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Nov 02 07:17:08.880143 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_user(user_id) Nov 02 07:17:08.880143 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Nov 02 07:17:08.880143 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Nov 02 07:17:08.881760 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:17:08.881760 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:17:08.881760 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Nov 02 07:17:08.881760 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:17:08.881760 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Nov 02 07:17:08.881760 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:17:08.881760 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:17:08.881760 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:17:08.881760 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:17:08.881760 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:17:08.881760 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:17:08.881760 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:17:08.881760 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:17:08.881760 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:17:08.881760 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:17:08.881760 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:17:08.881760 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:17:08.881760 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:17:08.882892 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:17:08.882892 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:17:08.882892 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Nov 02 07:17:08.882892 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Nov 02 07:17:08.882892 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Nov 02 07:17:08.882892 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Nov 02 07:17:08.882892 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Nov 02 07:17:08.882892 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Nov 02 07:17:08.882892 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Nov 02 07:17:08.882892 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:17:08.882892 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 195/383] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Nov 2 07:17:08 2022] GET /identity/v3/users/glance => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:17:08.883777 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3595108-85ea-445a-9038-758426a6d4b4 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:08.893565 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d0b909-328f-42aa-bb74-36451edbc7d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:08.894196 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d0b909-328f-42aa-bb74-36451edbc7d6 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:08.894514 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d0b909-328f-42aa-bb74-36451edbc7d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:08.894786 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d0b909-328f-42aa-bb74-36451edbc7d6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:08.906169 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d0b909-328f-42aa-bb74-36451edbc7d6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:17:08.911095 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-83d0b909-328f-42aa-bb74-36451edbc7d6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Nov 02 07:17:08.911095 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:17:08.911095 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:17:08.911095 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:17:08.911095 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:17:08.911095 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:17:08.911095 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:17:08.911095 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:17:08.911095 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:17:08.911095 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:17:08.911095 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:17:08.911095 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:17:08.911095 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:17:08.911095 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:17:08.911095 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:17:08.911095 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:17:08.912319 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:17:08.912319 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:17:08.912319 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:17:08.912319 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:17:08.912319 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:17:08.912319 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:17:08.912319 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:17:08.912319 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:17:08.912319 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:17:08.912319 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:17:08.912319 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:17:08.912319 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Nov 02 07:17:08.912319 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:17:08.912319 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:17:08.912319 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:17:08.912319 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:17:08.913107 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:17:08.913107 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:17:08.913107 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:17:08.913107 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:17:08.913107 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:17:08.913107 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:17:08.913107 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:17:08.913107 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:17:08.913107 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:17:08.913107 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:17:08.913107 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Nov 02 07:17:08.913107 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Nov 02 07:17:08.913107 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 189/384] 10.222.0.18 () {56 vars in 1102 bytes} [Wed Nov 2 07:17:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:17:08.913775 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d0b909-328f-42aa-bb74-36451edbc7d6 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:08.983653 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3595108-85ea-445a-9038-758426a6d4b4 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:09.012371 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d0b909-328f-42aa-bb74-36451edbc7d6 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:09.016988 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940af13b-1edb-4de5-b8d1-79d8046e8dde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:09.017586 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940af13b-1edb-4de5-b8d1-79d8046e8dde None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:09.017864 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940af13b-1edb-4de5-b8d1-79d8046e8dde None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:09.018133 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940af13b-1edb-4de5-b8d1-79d8046e8dde None admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:09.018822 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940af13b-1edb-4de5-b8d1-79d8046e8dde None admin] RBAC: Adding query filter params (name=service) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:17:09.019119 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940af13b-1edb-4de5-b8d1-79d8046e8dde None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:09.020474 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940af13b-1edb-4de5-b8d1-79d8046e8dde None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:09.028355 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 196/385] 10.222.0.18 () {56 vars in 1103 bytes} [Wed Nov 2 07:17:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:17:09.029361 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e72d5c3-bae4-4a85-aa33-98d649f586c7 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:09.029643 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e72d5c3-bae4-4a85-aa33-98d649f586c7 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:09.029916 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e72d5c3-bae4-4a85-aa33-98d649f586c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:09.047227 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-3e72d5c3-bae4-4a85-aa33-98d649f586c7 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:17:09.048414 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-3e72d5c3-bae4-4a85-aa33-98d649f586c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:17:09.048877 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f052fb1-1c84-40df-a315-d98e6d532061 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:09.049480 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f052fb1-1c84-40df-a315-d98e6d532061 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:09.049760 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f052fb1-1c84-40df-a315-d98e6d532061 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:09.050032 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f052fb1-1c84-40df-a315-d98e6d532061 None admin] PATH_INFO: `/v3/domains` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:09.050698 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f052fb1-1c84-40df-a315-d98e6d532061 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:17:09.051001 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f052fb1-1c84-40df-a315-d98e6d532061 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:09.052930 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f052fb1-1c84-40df-a315-d98e6d532061 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:09.064649 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 190/386] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Nov 2 07:17:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:17:09.065433 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f052fb1-1c84-40df-a315-d98e6d532061 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:09.072166 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:09.081585 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:09.116460 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 197/387] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Nov 2 07:17:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:17:09.117284 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e72d5c3-bae4-4a85-aa33-98d649f586c7 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:09.176118 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f052fb1-1c84-40df-a315-d98e6d532061 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:09.213706 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499a4562-7452-4599-92e3-2412f0b4d5ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:09.214310 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499a4562-7452-4599-92e3-2412f0b4d5ad None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:09.214620 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499a4562-7452-4599-92e3-2412f0b4d5ad None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:09.214900 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499a4562-7452-4599-92e3-2412f0b4d5ad None admin] PATH_INFO: `/v3/users` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:09.215565 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e72d5c3-bae4-4a85-aa33-98d649f586c7 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:09.215657 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499a4562-7452-4599-92e3-2412f0b4d5ad None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:17:09.215911 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499a4562-7452-4599-92e3-2412f0b4d5ad None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:09.217326 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499a4562-7452-4599-92e3-2412f0b4d5ad None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:09.249515 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6859c604-2256-4bfe-8a74-bc3954c3ee92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:09.250245 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6859c604-2256-4bfe-8a74-bc3954c3ee92 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:09.250653 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6859c604-2256-4bfe-8a74-bc3954c3ee92 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:09.251025 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6859c604-2256-4bfe-8a74-bc3954c3ee92 None admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:09.251771 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6859c604-2256-4bfe-8a74-bc3954c3ee92 None admin] RBAC: Adding query filter params (name=service) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:17:09.252161 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6859c604-2256-4bfe-8a74-bc3954c3ee92 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:09.253618 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6859c604-2256-4bfe-8a74-bc3954c3ee92 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:09.259670 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 191/388] 10.222.0.18 () {56 vars in 1137 bytes} [Wed Nov 2 07:17:09 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:17:09.259670 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499a4562-7452-4599-92e3-2412f0b4d5ad None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:09.267309 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 198/389] 10.222.0.18 () {56 vars in 1103 bytes} [Wed Nov 2 07:17:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:17:09.272586 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6859c604-2256-4bfe-8a74-bc3954c3ee92 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:09.294189 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:09.332237 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:09.366165 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499a4562-7452-4599-92e3-2412f0b4d5ad None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:09.379736 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6859c604-2256-4bfe-8a74-bc3954c3ee92 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:09.403446 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da55e9bd-0b00-43d9-9a17-ffc6cb07e169 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:09.404075 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da55e9bd-0b00-43d9-9a17-ffc6cb07e169 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:09.404351 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da55e9bd-0b00-43d9-9a17-ffc6cb07e169 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:09.404615 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da55e9bd-0b00-43d9-9a17-ffc6cb07e169 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:09.413769 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf0aab4-3c36-4730-8a73-609e36a79f78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:09.414444 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf0aab4-3c36-4730-8a73-609e36a79f78 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:09.414721 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf0aab4-3c36-4730-8a73-609e36a79f78 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:09.414876 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da55e9bd-0b00-43d9-9a17-ffc6cb07e169 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:09.414979 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf0aab4-3c36-4730-8a73-609e36a79f78 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:09.416313 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da55e9bd-0b00-43d9-9a17-ffc6cb07e169 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:09.421348 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf0aab4-3c36-4730-8a73-609e36a79f78 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:09.422806 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf0aab4-3c36-4730-8a73-609e36a79f78 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:09.431208 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-da55e9bd-0b00-43d9-9a17-ffc6cb07e169 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:17:09.431208 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:17:09.431208 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:17:09.431208 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:17:09.431208 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:17:09.431208 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:17:09.431208 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:17:09.431208 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:17:09.431208 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:17:09.431208 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:17:09.431208 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:17:09.431208 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:17:09.431208 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:17:09.431208 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:17:09.431208 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:17:09.431208 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:17:09.432279 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:17:09.432279 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:17:09.432279 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:17:09.432279 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:17:09.432279 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:17:09.432279 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:17:09.432279 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:17:09.432279 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Nov 02 07:17:09.432279 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:17:09.432279 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:17:09.432279 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:17:09.432279 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:17:09.432279 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:17:09.432279 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:17:09.432279 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:17:09.432279 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:17:09.432279 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:17:09.433378 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:17:09.433378 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:17:09.433378 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Nov 02 07:17:09.433378 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 192/390] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Nov 2 07:17:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:17:09.433378 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da55e9bd-0b00-43d9-9a17-ffc6cb07e169 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:09.435392 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-2bf0aab4-3c36-4730-8a73-609e36a79f78 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 07:17:09.435392 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:17:09.435392 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:17:09.435392 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:17:09.435392 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:17:09.435392 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:17:09.435392 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:17:09.435392 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:17:09.435392 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:17:09.435392 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:17:09.435392 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:17:09.435392 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:17:09.435392 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Nov 02 07:17:09.435392 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_project(project_id) Nov 02 07:17:09.435392 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Nov 02 07:17:09.436229 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Nov 02 07:17:09.436229 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:17:09.436229 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:17:09.436229 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:17:09.436229 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:17:09.436229 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:17:09.436229 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:17:09.436229 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:17:09.436229 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:17:09.436229 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:17:09.436229 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:17:09.436229 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:17:09.436229 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:17:09.436229 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:17:09.436229 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:17:09.436229 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:17:09.436229 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:17:09.437049 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Nov 02 07:17:09.437049 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Nov 02 07:17:09.437049 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Nov 02 07:17:09.437049 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Nov 02 07:17:09.437049 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Nov 02 07:17:09.437049 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Nov 02 07:17:09.437049 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 07:17:09.437049 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:17:09.437049 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 199/391] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Nov 2 07:17:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:17:09.437508 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf0aab4-3c36-4730-8a73-609e36a79f78 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:09.517712 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:09.535929 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da55e9bd-0b00-43d9-9a17-ffc6cb07e169 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:09.536485 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf0aab4-3c36-4730-8a73-609e36a79f78 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:09.559943 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:09.570015 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d6539f-fe90-4a10-8e91-4965971c74e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:09.570672 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d6539f-fe90-4a10-8e91-4965971c74e6 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:09.570866 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f389e30-2b6d-4382-998d-dee2ae44318e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:09.571061 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d6539f-fe90-4a10-8e91-4965971c74e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:09.571201 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d6539f-fe90-4a10-8e91-4965971c74e6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:09.571494 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f389e30-2b6d-4382-998d-dee2ae44318e None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:09.571778 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f389e30-2b6d-4382-998d-dee2ae44318e None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:09.572053 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f389e30-2b6d-4382-998d-dee2ae44318e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:09.581674 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d6539f-fe90-4a10-8e91-4965971c74e6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:09.582290 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f389e30-2b6d-4382-998d-dee2ae44318e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:09.583125 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d6539f-fe90-4a10-8e91-4965971c74e6 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:09.583788 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f389e30-2b6d-4382-998d-dee2ae44318e None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:09.596897 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-49d6539f-fe90-4a10-8e91-4965971c74e6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:17:09.596897 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:17:09.596897 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:17:09.596897 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:17:09.596897 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:17:09.596897 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:17:09.596897 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:17:09.596897 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:17:09.596897 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:17:09.596897 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:17:09.596897 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:17:09.596897 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:17:09.596897 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:17:09.596897 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:17:09.596897 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:17:09.596897 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:17:09.598101 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:17:09.598101 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:17:09.598101 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:17:09.598101 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:17:09.598101 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:17:09.598101 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:17:09.598101 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:17:09.598101 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:17:09.598101 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:17:09.598101 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:17:09.598101 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:17:09.598101 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:17:09.598101 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:17:09.598101 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:17:09.598101 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:17:09.598101 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:17:09.598101 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:17:09.600084 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:17:09.600084 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:17:09.600084 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:17:09.600250 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-2f389e30-2b6d-4382-998d-dee2ae44318e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:17:09.600250 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:17:09.600250 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:17:09.600250 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:17:09.600250 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:17:09.600250 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:17:09.600250 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:17:09.600250 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:17:09.600250 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:17:09.600250 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:17:09.600250 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:17:09.600250 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:17:09.600250 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:17:09.600250 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:17:09.600250 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:17:09.600250 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:17:09.601213 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:17:09.601213 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:17:09.601213 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:17:09.601213 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:17:09.601213 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:17:09.601213 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:17:09.601213 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:17:09.601213 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Nov 02 07:17:09.601213 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:17:09.601213 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:17:09.601213 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:17:09.601213 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:17:09.601213 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:17:09.601213 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:17:09.601213 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:17:09.601213 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:17:09.601213 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:17:09.602037 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:17:09.602037 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:17:09.602037 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Nov 02 07:17:09.602192 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 200/392] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Nov 2 07:17:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:17:09.602192 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d6539f-fe90-4a10-8e91-4965971c74e6 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:09.602324 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 193/393] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Nov 2 07:17:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:17:09.602324 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f389e30-2b6d-4382-998d-dee2ae44318e None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:09.702748 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f389e30-2b6d-4382-998d-dee2ae44318e None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:09.705005 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d6539f-fe90-4a10-8e91-4965971c74e6 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:09.734753 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:09.736839 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75671303-76e7-4cd3-bcb3-eae06cef340e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:09.737439 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75671303-76e7-4cd3-bcb3-eae06cef340e None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:09.737720 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75671303-76e7-4cd3-bcb3-eae06cef340e None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:09.737990 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75671303-76e7-4cd3-bcb3-eae06cef340e None admin] PATH_INFO: `/v3/domains` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:09.738649 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75671303-76e7-4cd3-bcb3-eae06cef340e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:17:09.738992 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75671303-76e7-4cd3-bcb3-eae06cef340e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:09.740044 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4945579-8dd1-4a14-8cb5-d7419a32fbb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:09.740511 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75671303-76e7-4cd3-bcb3-eae06cef340e None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:09.740657 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4945579-8dd1-4a14-8cb5-d7419a32fbb3 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:09.740931 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4945579-8dd1-4a14-8cb5-d7419a32fbb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:09.741196 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4945579-8dd1-4a14-8cb5-d7419a32fbb3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:09.751466 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4945579-8dd1-4a14-8cb5-d7419a32fbb3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:09.752094 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 194/394] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Nov 2 07:17:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:17:09.752682 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75671303-76e7-4cd3-bcb3-eae06cef340e None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:09.752900 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4945579-8dd1-4a14-8cb5-d7419a32fbb3 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:09.769566 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-b4945579-8dd1-4a14-8cb5-d7419a32fbb3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:17:09.769566 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:17:09.769566 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:17:09.769566 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:17:09.769566 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:17:09.769566 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:17:09.769566 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:17:09.769566 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:17:09.769566 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:17:09.769566 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:17:09.769566 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:17:09.769566 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:17:09.769566 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:17:09.769566 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:17:09.769566 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:17:09.769566 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:17:09.770652 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:17:09.770652 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:17:09.770652 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:17:09.770652 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:17:09.770652 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:17:09.770652 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:17:09.770652 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:17:09.770652 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:17:09.770652 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:17:09.770652 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:17:09.770652 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:17:09.770652 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:17:09.770652 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:17:09.770652 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:17:09.770652 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:17:09.770652 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:17:09.770652 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:17:09.772689 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:17:09.772689 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:17:09.772689 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:17:09.772689 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 201/395] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Nov 2 07:17:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:17:09.775247 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4945579-8dd1-4a14-8cb5-d7419a32fbb3 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:09.785626 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:09.849941 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75671303-76e7-4cd3-bcb3-eae06cef340e None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:09.873507 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4945579-8dd1-4a14-8cb5-d7419a32fbb3 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:09.883057 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd91c16b-e94f-45e8-81ac-60f409cf2cdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:09.883655 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd91c16b-e94f-45e8-81ac-60f409cf2cdf None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:09.883929 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd91c16b-e94f-45e8-81ac-60f409cf2cdf None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:09.884196 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd91c16b-e94f-45e8-81ac-60f409cf2cdf None admin] PATH_INFO: `/v3/projects` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:09.884867 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd91c16b-e94f-45e8-81ac-60f409cf2cdf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:17:09.885166 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd91c16b-e94f-45e8-81ac-60f409cf2cdf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:09.886601 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd91c16b-e94f-45e8-81ac-60f409cf2cdf None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:09.903808 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 195/396] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Nov 2 07:17:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:17:09.904745 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd91c16b-e94f-45e8-81ac-60f409cf2cdf None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:09.906158 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3a50a4-95f5-4bbd-af2b-06154476a7d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:09.906791 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3a50a4-95f5-4bbd-af2b-06154476a7d7 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:09.907075 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3a50a4-95f5-4bbd-af2b-06154476a7d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:09.907348 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3a50a4-95f5-4bbd-af2b-06154476a7d7 None admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:09.907987 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3a50a4-95f5-4bbd-af2b-06154476a7d7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:17:09.908302 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3a50a4-95f5-4bbd-af2b-06154476a7d7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:09.909642 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3a50a4-95f5-4bbd-af2b-06154476a7d7 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:09.921398 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 202/397] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Nov 2 07:17:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:17:09.922205 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3a50a4-95f5-4bbd-af2b-06154476a7d7 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:09.968096 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:09.999229 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:10.006633 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd91c16b-e94f-45e8-81ac-60f409cf2cdf None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:10.021704 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3a50a4-95f5-4bbd-af2b-06154476a7d7 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:10.039043 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c052f42-745b-4a21-8fd5-c0ee46ccdfa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:10.039644 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c052f42-745b-4a21-8fd5-c0ee46ccdfa5 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:10.039924 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c052f42-745b-4a21-8fd5-c0ee46ccdfa5 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:10.040195 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c052f42-745b-4a21-8fd5-c0ee46ccdfa5 None admin] PATH_INFO: `/v3/domains` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:10.040826 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c052f42-745b-4a21-8fd5-c0ee46ccdfa5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:17:10.041131 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c052f42-745b-4a21-8fd5-c0ee46ccdfa5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:10.042534 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c052f42-745b-4a21-8fd5-c0ee46ccdfa5 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:10.053975 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 196/398] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Nov 2 07:17:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:17:10.054582 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c052f42-745b-4a21-8fd5-c0ee46ccdfa5 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:10.055632 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cd071a-b21c-4b16-843c-8d98eea75f58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:10.056350 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cd071a-b21c-4b16-843c-8d98eea75f58 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:10.056741 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cd071a-b21c-4b16-843c-8d98eea75f58 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:10.057025 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cd071a-b21c-4b16-843c-8d98eea75f58 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:10.064087 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cd071a-b21c-4b16-843c-8d98eea75f58 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:10.065499 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cd071a-b21c-4b16-843c-8d98eea75f58 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:10.078436 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-b5cd071a-b21c-4b16-843c-8d98eea75f58 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Nov 02 07:17:10.078436 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:17:10.078436 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:17:10.078436 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:17:10.078436 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:17:10.078436 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:17:10.078436 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:17:10.078436 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:17:10.078436 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:17:10.078436 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:17:10.078436 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:17:10.078436 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:17:10.078436 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Nov 02 07:17:10.078436 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_user(user_id) Nov 02 07:17:10.078436 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Nov 02 07:17:10.078436 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Nov 02 07:17:10.079480 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:17:10.079480 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:17:10.079480 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Nov 02 07:17:10.079480 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:17:10.079480 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Nov 02 07:17:10.079480 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:17:10.079480 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:17:10.079480 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:17:10.079480 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:17:10.079480 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:17:10.079480 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:17:10.079480 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:17:10.079480 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:17:10.079480 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:17:10.079480 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:17:10.079480 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:17:10.079480 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:17:10.079480 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:17:10.080717 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:17:10.080717 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:17:10.080717 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Nov 02 07:17:10.080717 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Nov 02 07:17:10.080717 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Nov 02 07:17:10.080717 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Nov 02 07:17:10.080717 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Nov 02 07:17:10.080717 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Nov 02 07:17:10.080717 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Nov 02 07:17:10.080717 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:17:10.080717 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 203/399] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Nov 2 07:17:09 2022] GET /identity/v3/users/cinder => generated 84 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:17:10.080717 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cd071a-b21c-4b16-843c-8d98eea75f58 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:10.154945 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c052f42-745b-4a21-8fd5-c0ee46ccdfa5 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:10.178022 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cd071a-b21c-4b16-843c-8d98eea75f58 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:10.181122 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:10.189575 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e3d16b-0054-40c1-a326-df1ce9403d60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:10.190178 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e3d16b-0054-40c1-a326-df1ce9403d60 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:10.190484 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e3d16b-0054-40c1-a326-df1ce9403d60 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:10.190757 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e3d16b-0054-40c1-a326-df1ce9403d60 None admin] PATH_INFO: `/v3/domains` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:10.191387 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e3d16b-0054-40c1-a326-df1ce9403d60 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:17:10.191692 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e3d16b-0054-40c1-a326-df1ce9403d60 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:10.193186 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e3d16b-0054-40c1-a326-df1ce9403d60 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:10.204636 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 197/400] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Nov 2 07:17:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:17:10.205421 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e3d16b-0054-40c1-a326-df1ce9403d60 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:10.212884 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ae1963-a59a-45e9-9aee-775af672bbdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:10.213611 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ae1963-a59a-45e9-9aee-775af672bbdf None admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:10.213892 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ae1963-a59a-45e9-9aee-775af672bbdf None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:10.214158 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ae1963-a59a-45e9-9aee-775af672bbdf None admin] PATH_INFO: `/v3/projects/af04daa468b64160a6176dcab29502a7/users/69e48cba3f804cf08b2477f01ae95179/roles/4af6f4c514284b8cbddf3e02ec826378` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:10.222887 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:10.229328 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ae1963-a59a-45e9-9aee-775af672bbdf None admin] RBAC: Authorizing `identity:create_grant(project_id=af04daa468b64160a6176dcab29502a7, user_id=69e48cba3f804cf08b2477f01ae95179, role_id=4af6f4c514284b8cbddf3e02ec826378)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:10.230794 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ae1963-a59a-45e9-9aee-775af672bbdf None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:10.243166 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 204/401] 10.222.0.18 () {58 vars in 1439 bytes} [Wed Nov 2 07:17:10 2022] PUT /identity/v3/projects/af04daa468b64160a6176dcab29502a7/users/69e48cba3f804cf08b2477f01ae95179/roles/4af6f4c514284b8cbddf3e02ec826378 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:17:10.243945 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ae1963-a59a-45e9-9aee-775af672bbdf None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:10.325893 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e3d16b-0054-40c1-a326-df1ce9403d60 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:10.358627 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ae1963-a59a-45e9-9aee-775af672bbdf None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:10.363759 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ba746f-5376-424b-931b-52176c40d5d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:10.364542 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ba746f-5376-424b-931b-52176c40d5d3 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:10.365509 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ba746f-5376-424b-931b-52176c40d5d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:10.365873 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ba746f-5376-424b-931b-52176c40d5d3 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:10.374160 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ba746f-5376-424b-931b-52176c40d5d3 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:10.375770 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ba746f-5376-424b-931b-52176c40d5d3 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:10.388677 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-73ba746f-5376-424b-931b-52176c40d5d3 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Nov 02 07:17:10.388677 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:17:10.388677 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:17:10.388677 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:17:10.388677 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:17:10.388677 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:17:10.388677 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:17:10.388677 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:17:10.388677 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:17:10.388677 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:17:10.388677 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:17:10.388677 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:17:10.388677 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Nov 02 07:17:10.388677 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_user(user_id) Nov 02 07:17:10.388677 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Nov 02 07:17:10.388677 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Nov 02 07:17:10.389836 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:17:10.389836 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:17:10.389836 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Nov 02 07:17:10.389836 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:17:10.389836 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Nov 02 07:17:10.389836 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:17:10.389836 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:17:10.389836 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:17:10.389836 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:17:10.389836 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:17:10.389836 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:17:10.389836 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Nov 02 07:17:10.389836 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:17:10.389836 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:17:10.389836 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:17:10.389836 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:17:10.389836 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:17:10.389836 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:17:10.391104 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:17:10.391104 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:17:10.391104 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Nov 02 07:17:10.391104 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Nov 02 07:17:10.391104 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Nov 02 07:17:10.391104 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Nov 02 07:17:10.391104 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Nov 02 07:17:10.391104 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Nov 02 07:17:10.391104 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Nov 02 07:17:10.391104 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Nov 02 07:17:10.391104 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 198/402] 10.222.0.18 () {56 vars in 1102 bytes} [Wed Nov 2 07:17:10 2022] GET /identity/v3/users/neutron => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:17:10.391104 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ba746f-5376-424b-931b-52176c40d5d3 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:10.404627 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7aab97-013c-433f-a6b7-f01286d3f52f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:10.405495 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7aab97-013c-433f-a6b7-f01286d3f52f None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:10.405863 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7aab97-013c-433f-a6b7-f01286d3f52f None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:10.406270 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7aab97-013c-433f-a6b7-f01286d3f52f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:10.415044 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7aab97-013c-433f-a6b7-f01286d3f52f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:10.416892 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7aab97-013c-433f-a6b7-f01286d3f52f None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:10.428589 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-ce7aab97-013c-433f-a6b7-f01286d3f52f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 07:17:10.428589 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:17:10.428589 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:17:10.428589 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:17:10.428589 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:17:10.428589 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:17:10.428589 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:17:10.428589 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:17:10.428589 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:17:10.428589 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:17:10.428589 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:17:10.428589 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:17:10.428589 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Nov 02 07:17:10.428589 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_project(project_id) Nov 02 07:17:10.428589 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Nov 02 07:17:10.429628 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Nov 02 07:17:10.429628 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:17:10.429628 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:17:10.429628 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:17:10.429628 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:17:10.429628 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:17:10.429628 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:17:10.429628 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:17:10.429628 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:17:10.429628 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:17:10.429628 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:17:10.429628 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:17:10.429628 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:17:10.429628 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:17:10.429628 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:17:10.429628 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:17:10.429628 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:17:10.430742 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Nov 02 07:17:10.430742 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Nov 02 07:17:10.430742 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Nov 02 07:17:10.430742 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Nov 02 07:17:10.430742 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Nov 02 07:17:10.430742 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Nov 02 07:17:10.430742 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 07:17:10.430742 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:17:10.430742 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 205/403] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Nov 2 07:17:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:17:10.430742 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7aab97-013c-433f-a6b7-f01286d3f52f None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:10.447602 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:10.481882 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:10.507198 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ba746f-5376-424b-931b-52176c40d5d3 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:10.539958 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7aab97-013c-433f-a6b7-f01286d3f52f None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:10.542826 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b87b368-9cd8-48d6-9d37-1a2d0afb312d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:10.543441 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b87b368-9cd8-48d6-9d37-1a2d0afb312d None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:10.543722 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b87b368-9cd8-48d6-9d37-1a2d0afb312d None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:10.543992 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b87b368-9cd8-48d6-9d37-1a2d0afb312d None admin] PATH_INFO: `/v3/users` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:10.544704 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b87b368-9cd8-48d6-9d37-1a2d0afb312d None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:17:10.545006 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b87b368-9cd8-48d6-9d37-1a2d0afb312d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:10.546444 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b87b368-9cd8-48d6-9d37-1a2d0afb312d None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:10.566687 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 199/404] 10.222.0.18 () {56 vars in 1137 bytes} [Wed Nov 2 07:17:10 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:17:10.567516 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b87b368-9cd8-48d6-9d37-1a2d0afb312d None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:10.581356 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9879be23-d4ec-4fc0-b38c-a58b4ac513a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:10.581887 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9879be23-d4ec-4fc0-b38c-a58b4ac513a7 None admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:10.582163 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9879be23-d4ec-4fc0-b38c-a58b4ac513a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:10.582465 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9879be23-d4ec-4fc0-b38c-a58b4ac513a7 None admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:10.583110 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9879be23-d4ec-4fc0-b38c-a58b4ac513a7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:10.584982 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9879be23-d4ec-4fc0-b38c-a58b4ac513a7 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:10.608637 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 206/405] 10.222.0.18 () {60 vars in 1132 bytes} [Wed Nov 2 07:17:10 2022] POST /identity/v3/users => generated 247 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:17:10.609475 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9879be23-d4ec-4fc0-b38c-a58b4ac513a7 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:10.711675 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:10.717004 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b87b368-9cd8-48d6-9d37-1a2d0afb312d None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:10.754949 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9879be23-d4ec-4fc0-b38c-a58b4ac513a7 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:10.763813 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f699179f-60b9-4bf5-8922-e9abd989297d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:10.765122 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f699179f-60b9-4bf5-8922-e9abd989297d None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:10.765407 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f699179f-60b9-4bf5-8922-e9abd989297d None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:10.765679 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f699179f-60b9-4bf5-8922-e9abd989297d None admin] PATH_INFO: `/v3/users` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:10.766459 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f699179f-60b9-4bf5-8922-e9abd989297d None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:17:10.767279 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f699179f-60b9-4bf5-8922-e9abd989297d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:10.769265 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f699179f-60b9-4bf5-8922-e9abd989297d None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:10.772646 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:10.796799 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 200/406] 10.222.0.18 () {56 vars in 1139 bytes} [Wed Nov 2 07:17:10 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:17:10.797781 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f699179f-60b9-4bf5-8922-e9abd989297d None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:10.800079 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773beb6f-02e4-422a-b9ee-7225df5723d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:10.800750 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773beb6f-02e4-422a-b9ee-7225df5723d0 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:10.801097 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773beb6f-02e4-422a-b9ee-7225df5723d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:10.801426 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773beb6f-02e4-422a-b9ee-7225df5723d0 None admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:10.802756 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773beb6f-02e4-422a-b9ee-7225df5723d0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:17:10.803226 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773beb6f-02e4-422a-b9ee-7225df5723d0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:10.804663 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773beb6f-02e4-422a-b9ee-7225df5723d0 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:10.823705 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 207/407] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Nov 2 07:17:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:17:10.824613 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773beb6f-02e4-422a-b9ee-7225df5723d0 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:10.933772 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f699179f-60b9-4bf5-8922-e9abd989297d None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:10.962290 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773beb6f-02e4-422a-b9ee-7225df5723d0 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:10.973436 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3698e95-f85b-441b-8493-96233b9508f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:10.975158 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3698e95-f85b-441b-8493-96233b9508f5 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:10.975567 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3698e95-f85b-441b-8493-96233b9508f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:10.975847 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3698e95-f85b-441b-8493-96233b9508f5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:10.991095 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3698e95-f85b-441b-8493-96233b9508f5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:10.992549 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3698e95-f85b-441b-8493-96233b9508f5 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:10.996198 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:11.003333 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca58893-70b8-44d8-abdf-735cd1206775 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:11.003988 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca58893-70b8-44d8-abdf-735cd1206775 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:11.004264 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca58893-70b8-44d8-abdf-735cd1206775 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:11.004530 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca58893-70b8-44d8-abdf-735cd1206775 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:11.016116 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca58893-70b8-44d8-abdf-735cd1206775 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:11.017384 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-f3698e95-f85b-441b-8493-96233b9508f5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:17:11.017384 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:17:11.017384 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:17:11.017384 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:17:11.017384 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:17:11.017384 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:17:11.017384 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:17:11.017384 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:17:11.017384 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:17:11.017384 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:17:11.017384 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:17:11.017384 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:17:11.017384 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:17:11.017384 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:17:11.017384 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:17:11.017384 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:17:11.018304 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:17:11.018304 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:17:11.018304 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:17:11.018304 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:17:11.018304 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:17:11.018304 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:17:11.018304 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:17:11.018304 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Nov 02 07:17:11.018304 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:17:11.018304 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:17:11.018304 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:17:11.018304 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:17:11.018304 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:17:11.018304 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:17:11.018304 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:17:11.018304 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:17:11.018304 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:17:11.019211 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:17:11.019211 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:17:11.019211 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Nov 02 07:17:11.019377 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca58893-70b8-44d8-abdf-735cd1206775 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:11.022423 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 201/408] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Nov 2 07:17:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:17:11.023488 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3698e95-f85b-441b-8493-96233b9508f5 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:11.040574 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-5ca58893-70b8-44d8-abdf-735cd1206775 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:17:11.040574 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:17:11.040574 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:17:11.040574 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:17:11.040574 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:17:11.040574 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:17:11.040574 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:17:11.040574 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:17:11.040574 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:17:11.040574 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:17:11.040574 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:17:11.040574 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:17:11.040574 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:17:11.040574 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:17:11.040574 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:17:11.040574 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:17:11.041643 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:17:11.041643 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:17:11.041643 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:17:11.041643 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:17:11.041643 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:17:11.041643 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:17:11.041643 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:17:11.041643 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:17:11.041643 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:17:11.041643 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:17:11.041643 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:17:11.041643 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:17:11.041643 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:17:11.041643 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:17:11.041643 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:17:11.041643 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:17:11.041643 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:17:11.042531 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:17:11.042531 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:17:11.042531 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:17:11.042531 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 208/409] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Nov 2 07:17:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:17:11.042531 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca58893-70b8-44d8-abdf-735cd1206775 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:11.085396 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:11.145402 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3698e95-f85b-441b-8493-96233b9508f5 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:11.147976 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca58893-70b8-44d8-abdf-735cd1206775 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:11.185344 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40c7dcf-039e-408a-937f-3f8c1c98914b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:11.185862 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40c7dcf-039e-408a-937f-3f8c1c98914b None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:11.186136 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40c7dcf-039e-408a-937f-3f8c1c98914b None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:11.186499 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40c7dcf-039e-408a-937f-3f8c1c98914b None admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:11.187138 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40c7dcf-039e-408a-937f-3f8c1c98914b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:17:11.187434 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40c7dcf-039e-408a-937f-3f8c1c98914b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:11.188774 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40c7dcf-039e-408a-937f-3f8c1c98914b None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:11.190177 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86019ab-c85c-4ae7-9ae0-e27606140f22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:11.191026 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86019ab-c85c-4ae7-9ae0-e27606140f22 None admin] REQUEST_METHOD: `PUT` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:11.191445 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86019ab-c85c-4ae7-9ae0-e27606140f22 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:11.191851 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86019ab-c85c-4ae7-9ae0-e27606140f22 None admin] PATH_INFO: `/v3/projects/af04daa468b64160a6176dcab29502a7/users/c5353f2af0f44d668e2a269647aa1dc2/roles/4af6f4c514284b8cbddf3e02ec826378` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:11.201306 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 209/410] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Nov 2 07:17:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:17:11.206239 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40c7dcf-039e-408a-937f-3f8c1c98914b None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:11.207925 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86019ab-c85c-4ae7-9ae0-e27606140f22 None admin] RBAC: Authorizing `identity:create_grant(project_id=af04daa468b64160a6176dcab29502a7, user_id=c5353f2af0f44d668e2a269647aa1dc2, role_id=4af6f4c514284b8cbddf3e02ec826378)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:11.210332 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86019ab-c85c-4ae7-9ae0-e27606140f22 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:11.229553 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 202/411] 10.222.0.18 () {58 vars in 1439 bytes} [Wed Nov 2 07:17:11 2022] PUT /identity/v3/projects/af04daa468b64160a6176dcab29502a7/users/c5353f2af0f44d668e2a269647aa1dc2/roles/4af6f4c514284b8cbddf3e02ec826378 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:17:11.230859 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86019ab-c85c-4ae7-9ae0-e27606140f22 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:11.254990 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:11.321550 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40c7dcf-039e-408a-937f-3f8c1c98914b None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:11.339481 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:11.346567 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86019ab-c85c-4ae7-9ae0-e27606140f22 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:11.363321 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56383b11-3ebb-4ca2-a679-d001f75ac9b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:11.363939 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56383b11-3ebb-4ca2-a679-d001f75ac9b7 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:11.364547 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56383b11-3ebb-4ca2-a679-d001f75ac9b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:11.364547 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56383b11-3ebb-4ca2-a679-d001f75ac9b7 None admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:11.365946 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56383b11-3ebb-4ca2-a679-d001f75ac9b7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:17:11.366249 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56383b11-3ebb-4ca2-a679-d001f75ac9b7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:11.367747 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56383b11-3ebb-4ca2-a679-d001f75ac9b7 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:11.380980 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 210/412] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Nov 2 07:17:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:17:11.386325 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56383b11-3ebb-4ca2-a679-d001f75ac9b7 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:11.390391 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2874df17-fd5e-4f67-b07d-2eb97aefc708 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:11.391217 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2874df17-fd5e-4f67-b07d-2eb97aefc708 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:11.391669 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2874df17-fd5e-4f67-b07d-2eb97aefc708 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:11.392102 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2874df17-fd5e-4f67-b07d-2eb97aefc708 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:11.400467 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2874df17-fd5e-4f67-b07d-2eb97aefc708 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:11.402269 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2874df17-fd5e-4f67-b07d-2eb97aefc708 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:11.415707 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-2874df17-fd5e-4f67-b07d-2eb97aefc708 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 07:17:11.415707 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:17:11.415707 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:17:11.415707 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:17:11.415707 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:17:11.415707 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:17:11.415707 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:17:11.415707 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:17:11.415707 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:17:11.415707 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:17:11.415707 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:17:11.415707 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:17:11.415707 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Nov 02 07:17:11.415707 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_project(project_id) Nov 02 07:17:11.415707 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Nov 02 07:17:11.416749 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Nov 02 07:17:11.416749 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:17:11.416749 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:17:11.416749 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:17:11.416749 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:17:11.416749 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:17:11.416749 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:17:11.416749 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:17:11.416749 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Nov 02 07:17:11.416749 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:17:11.416749 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:17:11.416749 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:17:11.416749 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:17:11.416749 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:17:11.416749 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:17:11.416749 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:17:11.416749 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:17:11.417735 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Nov 02 07:17:11.417735 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Nov 02 07:17:11.417735 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Nov 02 07:17:11.417735 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Nov 02 07:17:11.417735 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Nov 02 07:17:11.417735 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Nov 02 07:17:11.417735 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 07:17:11.417735 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Nov 02 07:17:11.419918 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 203/413] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Nov 2 07:17:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:17:11.428714 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2874df17-fd5e-4f67-b07d-2eb97aefc708 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:11.501935 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56383b11-3ebb-4ca2-a679-d001f75ac9b7 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:11.503521 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:11.540060 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6459c8e-e16f-49a9-bfb8-ce5022cd5172 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:11.540705 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6459c8e-e16f-49a9-bfb8-ce5022cd5172 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:11.540999 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6459c8e-e16f-49a9-bfb8-ce5022cd5172 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:11.541233 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6459c8e-e16f-49a9-bfb8-ce5022cd5172 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:11.548406 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6459c8e-e16f-49a9-bfb8-ce5022cd5172 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:11.549695 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2874df17-fd5e-4f67-b07d-2eb97aefc708 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:11.551667 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6459c8e-e16f-49a9-bfb8-ce5022cd5172 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:11.572513 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-f6459c8e-e16f-49a9-bfb8-ce5022cd5172 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 07:17:11.572513 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:17:11.572513 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:17:11.572513 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:17:11.572513 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:17:11.572513 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:17:11.572513 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:17:11.572513 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:17:11.572513 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:17:11.572513 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:17:11.572513 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:17:11.572513 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:17:11.572513 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Nov 02 07:17:11.572513 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_project(project_id) Nov 02 07:17:11.572513 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Nov 02 07:17:11.573433 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Nov 02 07:17:11.573433 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:17:11.573433 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:17:11.573433 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:17:11.573433 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:17:11.573433 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:17:11.573433 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:17:11.573433 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:17:11.573433 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:17:11.573433 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:17:11.573433 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:17:11.573433 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:17:11.573433 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:17:11.573433 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:17:11.573433 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:17:11.573433 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:17:11.573433 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:17:11.574620 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Nov 02 07:17:11.574620 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Nov 02 07:17:11.574620 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Nov 02 07:17:11.574620 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Nov 02 07:17:11.574620 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Nov 02 07:17:11.574620 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Nov 02 07:17:11.574620 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 07:17:11.574620 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:17:11.574620 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 211/414] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Nov 2 07:17:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 07:17:11.579696 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6459c8e-e16f-49a9-bfb8-ce5022cd5172 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:11.594918 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48652ed-15d8-480f-8445-99ad0e2af3a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:11.596127 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48652ed-15d8-480f-8445-99ad0e2af3a6 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:11.596545 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48652ed-15d8-480f-8445-99ad0e2af3a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:11.596997 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48652ed-15d8-480f-8445-99ad0e2af3a6 None admin] PATH_INFO: `/v3/projects` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:11.597847 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48652ed-15d8-480f-8445-99ad0e2af3a6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:17:11.598301 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48652ed-15d8-480f-8445-99ad0e2af3a6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:11.600454 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48652ed-15d8-480f-8445-99ad0e2af3a6 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:11.613548 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 204/415] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Nov 2 07:17:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:17:11.621044 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48652ed-15d8-480f-8445-99ad0e2af3a6 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:11.649776 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:11.710828 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6459c8e-e16f-49a9-bfb8-ce5022cd5172 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:11.740150 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48652ed-15d8-480f-8445-99ad0e2af3a6 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:11.748512 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faaedd48-7435-4bcd-b9bf-57a9b0058b84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:11.749338 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faaedd48-7435-4bcd-b9bf-57a9b0058b84 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:11.749686 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faaedd48-7435-4bcd-b9bf-57a9b0058b84 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:11.750025 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faaedd48-7435-4bcd-b9bf-57a9b0058b84 None admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:11.750892 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faaedd48-7435-4bcd-b9bf-57a9b0058b84 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:17:11.751272 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faaedd48-7435-4bcd-b9bf-57a9b0058b84 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:11.753128 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faaedd48-7435-4bcd-b9bf-57a9b0058b84 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:11.768766 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 212/416] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Nov 2 07:17:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:17:11.776536 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faaedd48-7435-4bcd-b9bf-57a9b0058b84 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:11.795948 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a939b4cb-1c0a-4c5f-ba1f-54c6fb977738 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:11.796614 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a939b4cb-1c0a-4c5f-ba1f-54c6fb977738 None admin] REQUEST_METHOD: `PUT` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:11.796945 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a939b4cb-1c0a-4c5f-ba1f-54c6fb977738 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:11.797219 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a939b4cb-1c0a-4c5f-ba1f-54c6fb977738 None admin] PATH_INFO: `/v3/projects/af04daa468b64160a6176dcab29502a7/users/43bb326fb82e4d2c83fb2eaf91860dc8/roles/4af6f4c514284b8cbddf3e02ec826378` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:11.824295 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:11.824606 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a939b4cb-1c0a-4c5f-ba1f-54c6fb977738 None admin] RBAC: Authorizing `identity:create_grant(project_id=af04daa468b64160a6176dcab29502a7, user_id=43bb326fb82e4d2c83fb2eaf91860dc8, role_id=4af6f4c514284b8cbddf3e02ec826378)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:11.826119 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a939b4cb-1c0a-4c5f-ba1f-54c6fb977738 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:11.865506 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 205/417] 10.222.0.18 () {58 vars in 1439 bytes} [Wed Nov 2 07:17:11 2022] PUT /identity/v3/projects/af04daa468b64160a6176dcab29502a7/users/43bb326fb82e4d2c83fb2eaf91860dc8/roles/4af6f4c514284b8cbddf3e02ec826378 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:17:11.935554 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faaedd48-7435-4bcd-b9bf-57a9b0058b84 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:11.989001 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5edce2e-8d84-4b40-88cb-05ee0f3a87b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:11.989733 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5edce2e-8d84-4b40-88cb-05ee0f3a87b4 None admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:11.990013 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5edce2e-8d84-4b40-88cb-05ee0f3a87b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:11.990286 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5edce2e-8d84-4b40-88cb-05ee0f3a87b4 None admin] PATH_INFO: `/v3/projects/af04daa468b64160a6176dcab29502a7/users/b3ec6ecb85f94ef59762877d9ea33ed0/roles/4af6f4c514284b8cbddf3e02ec826378` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:12.014310 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5edce2e-8d84-4b40-88cb-05ee0f3a87b4 None admin] RBAC: Authorizing `identity:create_grant(project_id=af04daa468b64160a6176dcab29502a7, user_id=b3ec6ecb85f94ef59762877d9ea33ed0, role_id=4af6f4c514284b8cbddf3e02ec826378)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:12.015695 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5edce2e-8d84-4b40-88cb-05ee0f3a87b4 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:12.044942 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 213/418] 10.222.0.18 () {58 vars in 1439 bytes} [Wed Nov 2 07:17:11 2022] PUT /identity/v3/projects/af04daa468b64160a6176dcab29502a7/users/b3ec6ecb85f94ef59762877d9ea33ed0/roles/4af6f4c514284b8cbddf3e02ec826378 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 07:17:12.688411 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcab54da-5e59-49dc-9cfe-f6131f257baf None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:12.688949 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcab54da-5e59-49dc-9cfe-f6131f257baf None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:12.688949 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcab54da-5e59-49dc-9cfe-f6131f257baf None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:12.690292 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 206/419] 10.222.0.18 () {54 vars in 927 bytes} [Wed Nov 2 07:17:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:17:12.698310 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60471920-a53d-443a-8861-a14dda6314d1 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:12.698628 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60471920-a53d-443a-8861-a14dda6314d1 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:12.698902 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60471920-a53d-443a-8861-a14dda6314d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:12.727414 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-60471920-a53d-443a-8861-a14dda6314d1 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:17:12.728895 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-60471920-a53d-443a-8861-a14dda6314d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:17:12.820355 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 214/420] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Nov 2 07:17:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:17:13.144196 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcab54da-5e59-49dc-9cfe-f6131f257baf None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:13.225068 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:13.268782 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcab54da-5e59-49dc-9cfe-f6131f257baf None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:13.305327 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b2d87f-2835-420b-947c-f60829b3f72b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:13.305953 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b2d87f-2835-420b-947c-f60829b3f72b None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:13.306232 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b2d87f-2835-420b-947c-f60829b3f72b None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:13.306550 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b2d87f-2835-420b-947c-f60829b3f72b None admin] PATH_INFO: `/v3/roles/service` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:13.321178 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b2d87f-2835-420b-947c-f60829b3f72b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:17:13.326579 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-f7b2d87f-2835-420b-947c-f60829b3f72b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Nov 02 07:17:13.326579 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:17:13.326579 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:17:13.326579 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:17:13.326579 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:17:13.326579 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:17:13.326579 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:17:13.326579 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:17:13.326579 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:17:13.326579 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:17:13.326579 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:17:13.326579 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:17:13.326579 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:17:13.326579 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:17:13.326579 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:17:13.326579 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:17:13.328640 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:17:13.328640 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:17:13.328640 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:17:13.328640 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:17:13.328640 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:17:13.328640 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:17:13.328640 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:17:13.328640 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:17:13.328640 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:17:13.328640 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:17:13.328640 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:17:13.328640 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Nov 02 07:17:13.328640 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:17:13.328640 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:17:13.328640 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:17:13.328640 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:17:13.330624 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:17:13.330624 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:17:13.330624 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:17:13.330624 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:17:13.330624 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:17:13.330624 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:17:13.330624 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:17:13.330624 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:17:13.330624 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:17:13.330624 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:17:13.330624 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Nov 02 07:17:13.330624 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Nov 02 07:17:13.330624 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 207/421] 10.222.0.18 () {56 vars in 1102 bytes} [Wed Nov 2 07:17:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:17:13.336597 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd65e7b-096c-4546-9f42-033ae976abd9 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:13.336866 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd65e7b-096c-4546-9f42-033ae976abd9 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:13.337149 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd65e7b-096c-4546-9f42-033ae976abd9 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:13.339031 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 215/422] 10.222.0.18 () {54 vars in 927 bytes} [Wed Nov 2 07:17:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:17:13.339499 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b2d87f-2835-420b-947c-f60829b3f72b None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:13.346398 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfc423b-6bf4-49f7-93eb-635b5496ba3b None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:13.346398 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfc423b-6bf4-49f7-93eb-635b5496ba3b None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:13.346398 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfc423b-6bf4-49f7-93eb-635b5496ba3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:13.362391 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:13.371623 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-dcfc423b-6bf4-49f7-93eb-635b5496ba3b None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:17:13.372891 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-dcfc423b-6bf4-49f7-93eb-635b5496ba3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:17:13.485522 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b2d87f-2835-420b-947c-f60829b3f72b None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:13.510475 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:13.521618 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 216/423] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Nov 2 07:17:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Nov 02 07:17:13.528995 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade55360-b7c2-4a64-9545-7b5e9f946880 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:13.529724 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade55360-b7c2-4a64-9545-7b5e9f946880 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:13.530100 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade55360-b7c2-4a64-9545-7b5e9f946880 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:13.530524 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade55360-b7c2-4a64-9545-7b5e9f946880 None admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:13.531283 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade55360-b7c2-4a64-9545-7b5e9f946880 None admin] RBAC: Adding query filter params (name=service) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:17:13.531682 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade55360-b7c2-4a64-9545-7b5e9f946880 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:13.533152 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade55360-b7c2-4a64-9545-7b5e9f946880 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:13.542171 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 208/424] 10.222.0.18 () {56 vars in 1103 bytes} [Wed Nov 2 07:17:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:17:13.547828 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfc423b-6bf4-49f7-93eb-635b5496ba3b None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:13.660882 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfc423b-6bf4-49f7-93eb-635b5496ba3b None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:13.695807 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:13.704085 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0297ecf3-1c9f-40d9-8460-56768bb46569 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:13.707074 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0297ecf3-1c9f-40d9-8460-56768bb46569 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:13.707505 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0297ecf3-1c9f-40d9-8460-56768bb46569 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:13.707972 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0297ecf3-1c9f-40d9-8460-56768bb46569 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:13.719297 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0297ecf3-1c9f-40d9-8460-56768bb46569 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:13.720726 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0297ecf3-1c9f-40d9-8460-56768bb46569 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:13.741387 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-0297ecf3-1c9f-40d9-8460-56768bb46569 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:17:13.741387 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:17:13.741387 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:17:13.741387 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:17:13.741387 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:17:13.741387 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:17:13.741387 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:17:13.741387 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:17:13.741387 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:17:13.741387 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:17:13.741387 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:17:13.741387 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:17:13.741387 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:17:13.741387 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:17:13.741387 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:17:13.741387 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:17:13.742724 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:17:13.742724 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:17:13.742724 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:17:13.742724 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:17:13.742724 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:17:13.742724 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:17:13.742724 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:17:13.742724 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:17:13.742724 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:17:13.742724 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:17:13.742724 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:17:13.742724 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:17:13.742724 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:17:13.742724 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:17:13.742724 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:17:13.742724 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:17:13.742724 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:17:13.743558 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:17:13.743558 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:17:13.743558 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:17:13.746526 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 217/425] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Nov 2 07:17:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:17:13.752651 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade55360-b7c2-4a64-9545-7b5e9f946880 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:13.907826 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0297ecf3-1c9f-40d9-8460-56768bb46569 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:13.923465 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade55360-b7c2-4a64-9545-7b5e9f946880 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:13.971847 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628aa0e5-8260-4620-90da-a87279b18b5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:13.972517 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628aa0e5-8260-4620-90da-a87279b18b5b None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:13.972795 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628aa0e5-8260-4620-90da-a87279b18b5b None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:13.973118 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628aa0e5-8260-4620-90da-a87279b18b5b None admin] PATH_INFO: `/v3/domains` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:13.973767 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628aa0e5-8260-4620-90da-a87279b18b5b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:17:13.974691 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628aa0e5-8260-4620-90da-a87279b18b5b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:13.976255 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628aa0e5-8260-4620-90da-a87279b18b5b None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:13.993478 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 209/426] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Nov 2 07:17:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:17:13.999077 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628aa0e5-8260-4620-90da-a87279b18b5b None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:14.050728 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:14.085666 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0297ecf3-1c9f-40d9-8460-56768bb46569 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:14.107733 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628aa0e5-8260-4620-90da-a87279b18b5b None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:14.121479 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0c492b-a734-4da7-bdd8-6f43bdb8eac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:14.122304 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0c492b-a734-4da7-bdd8-6f43bdb8eac7 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:14.122781 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0c492b-a734-4da7-bdd8-6f43bdb8eac7 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:14.123198 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0c492b-a734-4da7-bdd8-6f43bdb8eac7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:14.134268 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0c492b-a734-4da7-bdd8-6f43bdb8eac7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:17:14.139769 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-6b0c492b-a734-4da7-bdd8-6f43bdb8eac7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Nov 02 07:17:14.139769 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:17:14.139769 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:17:14.139769 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:17:14.139769 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:17:14.139769 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:17:14.139769 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:17:14.139769 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:17:14.139769 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:17:14.139769 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:17:14.139769 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:17:14.139769 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:17:14.139769 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:17:14.139769 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:17:14.139769 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:17:14.139769 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:17:14.140642 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:17:14.140642 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:17:14.140642 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:17:14.140642 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:17:14.140642 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:17:14.140642 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:17:14.140642 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:17:14.140642 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:17:14.140642 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:17:14.140642 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:17:14.140642 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:17:14.140642 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:17:14.140642 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:17:14.140642 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:17:14.140642 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:17:14.140642 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:17:14.141427 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:17:14.141427 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:17:14.141427 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:17:14.141427 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:17:14.141427 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:17:14.141427 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:17:14.141427 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:17:14.141427 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:17:14.141427 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:17:14.141427 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:17:14.141427 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Nov 02 07:17:14.141427 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:17:14.143878 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 218/427] 10.222.0.18 () {56 vars in 1102 bytes} [Wed Nov 2 07:17:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 07:17:14.145199 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9628d5-4f79-41b8-ba7e-a81b23de9868 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:14.145607 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9628d5-4f79-41b8-ba7e-a81b23de9868 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:14.146059 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9628d5-4f79-41b8-ba7e-a81b23de9868 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:14.147718 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 219/428] 10.222.0.18 () {54 vars in 927 bytes} [Wed Nov 2 07:17:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:17:14.151070 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9628d5-4f79-41b8-ba7e-a81b23de9868 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:14.151427 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331a5592-5c00-4c29-8d96-56b3f8d4c317 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:14.152142 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331a5592-5c00-4c29-8d96-56b3f8d4c317 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:14.152569 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331a5592-5c00-4c29-8d96-56b3f8d4c317 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:14.152935 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331a5592-5c00-4c29-8d96-56b3f8d4c317 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:14.161297 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331a5592-5c00-4c29-8d96-56b3f8d4c317 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:14.163452 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331a5592-5c00-4c29-8d96-56b3f8d4c317 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:14.176169 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-331a5592-5c00-4c29-8d96-56b3f8d4c317 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Nov 02 07:17:14.176169 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:17:14.176169 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:17:14.176169 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:17:14.176169 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:17:14.176169 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:17:14.176169 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:17:14.176169 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:17:14.176169 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:17:14.176169 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:17:14.176169 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:17:14.176169 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:17:14.176169 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Nov 02 07:17:14.176169 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_user(user_id) Nov 02 07:17:14.176169 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Nov 02 07:17:14.176169 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Nov 02 07:17:14.177328 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:17:14.177328 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:17:14.177328 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Nov 02 07:17:14.177328 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:17:14.177328 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Nov 02 07:17:14.177328 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:17:14.177328 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:17:14.177328 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:17:14.177328 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:17:14.177328 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:17:14.177328 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:17:14.177328 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Nov 02 07:17:14.177328 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:17:14.177328 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:17:14.177328 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:17:14.177328 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:17:14.177328 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:17:14.177328 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:17:14.178553 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:17:14.178553 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:17:14.178553 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Nov 02 07:17:14.178553 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Nov 02 07:17:14.178553 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Nov 02 07:17:14.178553 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Nov 02 07:17:14.178553 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Nov 02 07:17:14.178553 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Nov 02 07:17:14.178553 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Nov 02 07:17:14.178553 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Nov 02 07:17:14.180003 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 210/429] 10.222.0.18 () {56 vars in 1093 bytes} [Wed Nov 2 07:17:13 2022] GET /identity/v3/users/nova => generated 82 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 07:17:14.181249 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16518bd8-cb54-475a-ad4f-144dbddf302c None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:14.181616 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16518bd8-cb54-475a-ad4f-144dbddf302c None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:14.182078 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16518bd8-cb54-475a-ad4f-144dbddf302c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:14.201114 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-16518bd8-cb54-475a-ad4f-144dbddf302c None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:17:14.202571 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-16518bd8-cb54-475a-ad4f-144dbddf302c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:17:14.206508 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:14.264771 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9628d5-4f79-41b8-ba7e-a81b23de9868 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:14.278698 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 211/430] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Nov 2 07:17:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:17:14.279932 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16518bd8-cb54-475a-ad4f-144dbddf302c None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:14.301317 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da23672d-61ad-43e8-9385-98973f9629b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:14.302937 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da23672d-61ad-43e8-9385-98973f9629b8 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:14.303396 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da23672d-61ad-43e8-9385-98973f9629b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:14.303844 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da23672d-61ad-43e8-9385-98973f9629b8 None admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:14.304677 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da23672d-61ad-43e8-9385-98973f9629b8 None admin] RBAC: Adding query filter params (name=service) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:17:14.305180 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da23672d-61ad-43e8-9385-98973f9629b8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:14.306805 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da23672d-61ad-43e8-9385-98973f9629b8 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:14.315610 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 220/431] 10.222.0.18 () {56 vars in 1103 bytes} [Wed Nov 2 07:17:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:17:14.321252 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da23672d-61ad-43e8-9385-98973f9629b8 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:14.323306 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:14.414550 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16518bd8-cb54-475a-ad4f-144dbddf302c None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:14.433959 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da23672d-61ad-43e8-9385-98973f9629b8 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:14.448163 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:14.471781 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31f2fde-666f-4ec0-8883-40290a5c3c1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:14.472905 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31f2fde-666f-4ec0-8883-40290a5c3c1c None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:14.473406 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31f2fde-666f-4ec0-8883-40290a5c3c1c None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:14.474051 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31f2fde-666f-4ec0-8883-40290a5c3c1c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:14.484820 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a9f89e-126e-4ba3-99a4-947ae764b0f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:14.485439 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a9f89e-126e-4ba3-99a4-947ae764b0f7 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:14.485722 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a9f89e-126e-4ba3-99a4-947ae764b0f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:14.485992 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a9f89e-126e-4ba3-99a4-947ae764b0f7 None admin] PATH_INFO: `/v3/users` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:14.486747 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a9f89e-126e-4ba3-99a4-947ae764b0f7 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:17:14.487049 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a9f89e-126e-4ba3-99a4-947ae764b0f7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:14.487732 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31f2fde-666f-4ec0-8883-40290a5c3c1c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:14.488480 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a9f89e-126e-4ba3-99a4-947ae764b0f7 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:14.490142 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31f2fde-666f-4ec0-8883-40290a5c3c1c None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:14.505826 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-d31f2fde-666f-4ec0-8883-40290a5c3c1c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:17:14.505826 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:17:14.505826 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:17:14.505826 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:17:14.505826 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:17:14.505826 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:17:14.505826 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:17:14.505826 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:17:14.505826 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:17:14.505826 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:17:14.505826 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:17:14.505826 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:17:14.505826 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:17:14.505826 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:17:14.505826 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:17:14.505826 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:17:14.506804 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:17:14.506804 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:17:14.506804 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:17:14.506804 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:17:14.506804 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:17:14.506804 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:17:14.506804 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:17:14.506804 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:17:14.506804 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:17:14.506804 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:17:14.506804 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:17:14.506804 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:17:14.506804 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:17:14.506804 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:17:14.506804 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:17:14.506804 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:17:14.506804 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:17:14.507665 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:17:14.507665 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:17:14.507665 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:17:14.509713 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 221/432] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Nov 2 07:17:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 07:17:14.522051 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 212/433] 10.222.0.18 () {56 vars in 1133 bytes} [Wed Nov 2 07:17:14 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:17:14.524346 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31f2fde-666f-4ec0-8883-40290a5c3c1c None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:14.524773 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a9f89e-126e-4ba3-99a4-947ae764b0f7 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:14.627187 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:14.634345 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a9f89e-126e-4ba3-99a4-947ae764b0f7 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:14.682132 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31f2fde-666f-4ec0-8883-40290a5c3c1c None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:14.685673 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85069f3-f738-4f87-add0-e8bd6c4594b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:14.686512 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85069f3-f738-4f87-add0-e8bd6c4594b3 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:14.687768 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85069f3-f738-4f87-add0-e8bd6c4594b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:14.688181 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85069f3-f738-4f87-add0-e8bd6c4594b3 None admin] PATH_INFO: `/v3/domains` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:14.689038 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85069f3-f738-4f87-add0-e8bd6c4594b3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:17:14.689607 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85069f3-f738-4f87-add0-e8bd6c4594b3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:14.691141 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85069f3-f738-4f87-add0-e8bd6c4594b3 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:14.713391 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 213/434] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Nov 2 07:17:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:17:14.713391 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85069f3-f738-4f87-add0-e8bd6c4594b3 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:14.745092 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c619aa-00e2-450d-8e23-c13cc45a47d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:14.745767 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c619aa-00e2-450d-8e23-c13cc45a47d7 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:14.746304 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c619aa-00e2-450d-8e23-c13cc45a47d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:14.746633 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c619aa-00e2-450d-8e23-c13cc45a47d7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:14.778109 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:14.822547 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c619aa-00e2-450d-8e23-c13cc45a47d7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:14.824093 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c619aa-00e2-450d-8e23-c13cc45a47d7 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:14.825036 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85069f3-f738-4f87-add0-e8bd6c4594b3 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:14.866807 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81363a0b-e2cc-4728-bc8f-04e1caeb9482 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:14.867427 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81363a0b-e2cc-4728-bc8f-04e1caeb9482 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:14.867704 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81363a0b-e2cc-4728-bc8f-04e1caeb9482 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:14.867971 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81363a0b-e2cc-4728-bc8f-04e1caeb9482 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:14.878323 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81363a0b-e2cc-4728-bc8f-04e1caeb9482 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:17:14.884868 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-81363a0b-e2cc-4728-bc8f-04e1caeb9482 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Nov 02 07:17:14.884868 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:17:14.884868 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:17:14.884868 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:17:14.884868 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:17:14.884868 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:17:14.884868 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:17:14.884868 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:17:14.884868 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:17:14.884868 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:17:14.884868 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:17:14.884868 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:17:14.884868 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:17:14.884868 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:17:14.884868 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:17:14.884868 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:17:14.885798 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:17:14.885798 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:17:14.885798 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:17:14.885798 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:17:14.885798 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:17:14.885798 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:17:14.885798 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:17:14.885798 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:17:14.885798 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:17:14.885798 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:17:14.885798 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:17:14.885798 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Nov 02 07:17:14.885798 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:17:14.885798 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:17:14.885798 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:17:14.885798 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:17:14.886608 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:17:14.886608 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:17:14.886608 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:17:14.886608 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:17:14.886608 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:17:14.886608 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:17:14.886608 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:17:14.886608 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:17:14.886608 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:17:14.886608 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:17:14.886608 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Nov 02 07:17:14.886608 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Nov 02 07:17:14.891993 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 214/435] 10.222.0.18 () {56 vars in 1102 bytes} [Wed Nov 2 07:17:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:17:14.892864 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81363a0b-e2cc-4728-bc8f-04e1caeb9482 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:14.909769 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:14.937598 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 222/436] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Nov 2 07:17:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Nov 02 07:17:14.938885 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7f22de-84c2-4e2f-b1af-1bf422bec813 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:14.939166 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7f22de-84c2-4e2f-b1af-1bf422bec813 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:14.939435 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7f22de-84c2-4e2f-b1af-1bf422bec813 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:14.940779 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 223/437] 10.222.0.18 () {54 vars in 927 bytes} [Wed Nov 2 07:17:14 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:17:14.941312 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7f22de-84c2-4e2f-b1af-1bf422bec813 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:14.998385 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81363a0b-e2cc-4728-bc8f-04e1caeb9482 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:14.998976 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:15.032705 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5ae91e-2bc5-4698-b7b8-7813fcee49c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:15.033241 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5ae91e-2bc5-4698-b7b8-7813fcee49c4 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:15.033514 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5ae91e-2bc5-4698-b7b8-7813fcee49c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:15.033777 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5ae91e-2bc5-4698-b7b8-7813fcee49c4 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:15.041525 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7f22de-84c2-4e2f-b1af-1bf422bec813 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:15.041642 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5ae91e-2bc5-4698-b7b8-7813fcee49c4 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:15.042989 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5ae91e-2bc5-4698-b7b8-7813fcee49c4 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:15.053758 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-5d5ae91e-2bc5-4698-b7b8-7813fcee49c4 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Nov 02 07:17:15.053758 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:17:15.053758 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:17:15.053758 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:17:15.053758 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:17:15.053758 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:17:15.053758 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:17:15.053758 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:17:15.053758 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:17:15.053758 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:17:15.053758 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:17:15.053758 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:17:15.053758 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Nov 02 07:17:15.053758 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_user(user_id) Nov 02 07:17:15.053758 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Nov 02 07:17:15.053758 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Nov 02 07:17:15.054719 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:17:15.054719 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:17:15.054719 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Nov 02 07:17:15.054719 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:17:15.054719 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Nov 02 07:17:15.054719 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:17:15.054719 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:17:15.054719 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:17:15.054719 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:17:15.054719 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:17:15.054719 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:17:15.054719 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Nov 02 07:17:15.054719 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:17:15.054719 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:17:15.054719 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:17:15.054719 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:17:15.054719 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:17:15.054719 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:17:15.055753 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:17:15.055753 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:17:15.055753 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Nov 02 07:17:15.055753 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Nov 02 07:17:15.055753 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Nov 02 07:17:15.055753 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Nov 02 07:17:15.055753 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Nov 02 07:17:15.055753 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Nov 02 07:17:15.055753 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Nov 02 07:17:15.055753 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Nov 02 07:17:15.055753 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 215/438] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Nov 2 07:17:14 2022] GET /identity/v3/users/swift => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:17:15.056461 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaa59fc-f62c-4389-b1d5-f1b859f72765 None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:15.056461 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaa59fc-f62c-4389-b1d5-f1b859f72765 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:15.056461 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaa59fc-f62c-4389-b1d5-f1b859f72765 None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:15.057809 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 216/439] 10.222.0.18 () {54 vars in 927 bytes} [Wed Nov 2 07:17:15 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:17:15.059520 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acaa59fc-f62c-4389-b1d5-f1b859f72765 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:15.078547 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e26912-8d84-4c1b-9e07-9b19f247c194 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:15.079124 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e26912-8d84-4c1b-9e07-9b19f247c194 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:15.079399 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e26912-8d84-4c1b-9e07-9b19f247c194 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:15.079664 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e26912-8d84-4c1b-9e07-9b19f247c194 None admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:15.080301 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e26912-8d84-4c1b-9e07-9b19f247c194 None admin] RBAC: Adding query filter params (name=service) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:17:15.080592 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e26912-8d84-4c1b-9e07-9b19f247c194 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:15.081940 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e26912-8d84-4c1b-9e07-9b19f247c194 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:15.090232 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 224/440] 10.222.0.18 () {56 vars in 1103 bytes} [Wed Nov 2 07:17:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:17:15.091380 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d65e61-2743-4b28-a5c3-a696e9f8e909 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:15.091661 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d65e61-2743-4b28-a5c3-a696e9f8e909 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:15.091930 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d65e61-2743-4b28-a5c3-a696e9f8e909 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:15.110173 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-39d65e61-2743-4b28-a5c3-a696e9f8e909 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:17:15.111373 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-39d65e61-2743-4b28-a5c3-a696e9f8e909 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:17:15.139601 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:15.165856 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acaa59fc-f62c-4389-b1d5-f1b859f72765 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:15.176120 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 225/441] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Nov 2 07:17:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:17:15.176942 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d65e61-2743-4b28-a5c3-a696e9f8e909 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:15.200228 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9f777f-b43e-4743-ba2c-243639094e96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:15.200851 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9f777f-b43e-4743-ba2c-243639094e96 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:15.201128 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9f777f-b43e-4743-ba2c-243639094e96 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:15.201389 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9f777f-b43e-4743-ba2c-243639094e96 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:15.213284 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9f777f-b43e-4743-ba2c-243639094e96 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:15.214754 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9f777f-b43e-4743-ba2c-243639094e96 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:15.228949 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:15.229852 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-5a9f777f-b43e-4743-ba2c-243639094e96 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:17:15.229852 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:17:15.229852 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:17:15.229852 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:17:15.229852 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:17:15.229852 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:17:15.229852 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:17:15.229852 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:17:15.229852 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:17:15.229852 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:17:15.229852 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:17:15.229852 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:17:15.229852 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:17:15.229852 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:17:15.229852 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:17:15.229852 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:17:15.230976 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:17:15.230976 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:17:15.230976 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:17:15.230976 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:17:15.230976 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:17:15.230976 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:17:15.230976 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:17:15.230976 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Nov 02 07:17:15.230976 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:17:15.230976 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:17:15.230976 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:17:15.230976 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:17:15.230976 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:17:15.230976 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:17:15.230976 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:17:15.230976 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:17:15.230976 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:17:15.231822 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:17:15.231822 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:17:15.231822 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Nov 02 07:17:15.231822 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 217/442] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Nov 2 07:17:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:17:15.232052 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a095af-e636-4214-8dc5-6d16b38d4c0d None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:15.232449 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a095af-e636-4214-8dc5-6d16b38d4c0d None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:15.232606 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a095af-e636-4214-8dc5-6d16b38d4c0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:15.250825 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-62a095af-e636-4214-8dc5-6d16b38d4c0d None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:17:15.252027 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-62a095af-e636-4214-8dc5-6d16b38d4c0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:17:15.287752 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d65e61-2743-4b28-a5c3-a696e9f8e909 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:15.330384 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 218/443] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Nov 2 07:17:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:17:15.330384 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a095af-e636-4214-8dc5-6d16b38d4c0d None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:15.332975 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0504557b-c343-4734-abb7-95efd5794a36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:15.333775 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0504557b-c343-4734-abb7-95efd5794a36 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:15.334212 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0504557b-c343-4734-abb7-95efd5794a36 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:15.334674 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0504557b-c343-4734-abb7-95efd5794a36 None admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:15.335521 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0504557b-c343-4734-abb7-95efd5794a36 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:17:15.335912 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0504557b-c343-4734-abb7-95efd5794a36 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:15.337699 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0504557b-c343-4734-abb7-95efd5794a36 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:15.361451 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 226/444] 10.222.0.18 () {56 vars in 1135 bytes} [Wed Nov 2 07:17:15 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:17:15.361451 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0504557b-c343-4734-abb7-95efd5794a36 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:15.402460 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:15.458551 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a095af-e636-4214-8dc5-6d16b38d4c0d None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:15.469838 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0504557b-c343-4734-abb7-95efd5794a36 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:15.479940 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:15.494344 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c522b6f1-e861-4b32-b83e-04040276b12e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:15.495111 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c522b6f1-e861-4b32-b83e-04040276b12e None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:15.495504 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c522b6f1-e861-4b32-b83e-04040276b12e None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:15.495871 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c522b6f1-e861-4b32-b83e-04040276b12e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:15.506683 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c522b6f1-e861-4b32-b83e-04040276b12e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:15.507533 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775d2bb4-3af2-45bf-8919-a53228cfd536 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:15.508129 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c522b6f1-e861-4b32-b83e-04040276b12e None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:15.508228 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775d2bb4-3af2-45bf-8919-a53228cfd536 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:15.508401 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775d2bb4-3af2-45bf-8919-a53228cfd536 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:15.508665 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775d2bb4-3af2-45bf-8919-a53228cfd536 None admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:15.509291 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775d2bb4-3af2-45bf-8919-a53228cfd536 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:17:15.509584 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775d2bb4-3af2-45bf-8919-a53228cfd536 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:15.511041 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775d2bb4-3af2-45bf-8919-a53228cfd536 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:15.526992 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 227/445] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Nov 2 07:17:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:17:15.527175 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-c522b6f1-e861-4b32-b83e-04040276b12e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:17:15.527175 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:17:15.527175 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:17:15.527175 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:17:15.527175 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:17:15.527175 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:17:15.527175 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:17:15.527175 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:17:15.527175 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:17:15.527175 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:17:15.527175 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:17:15.527175 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:17:15.527175 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:17:15.527175 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:17:15.527175 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:17:15.527175 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:17:15.527969 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:17:15.527969 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:17:15.527969 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:17:15.527969 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:17:15.527969 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:17:15.527969 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:17:15.527969 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:17:15.527969 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Nov 02 07:17:15.527969 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:17:15.527969 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:17:15.527969 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:17:15.527969 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:17:15.527969 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:17:15.527969 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:17:15.527969 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:17:15.527969 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:17:15.527969 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:17:15.528801 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:17:15.528801 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:17:15.528801 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Nov 02 07:17:15.528971 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775d2bb4-3af2-45bf-8919-a53228cfd536 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:15.529066 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 219/446] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Nov 2 07:17:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:17:15.529066 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c522b6f1-e861-4b32-b83e-04040276b12e None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:15.634228 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c522b6f1-e861-4b32-b83e-04040276b12e None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:15.634875 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775d2bb4-3af2-45bf-8919-a53228cfd536 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:15.645224 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:15.669058 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2710de30-c86e-4bd6-a6e4-9ccf162b6e8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:15.669682 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2710de30-c86e-4bd6-a6e4-9ccf162b6e8d None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:15.669954 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2710de30-c86e-4bd6-a6e4-9ccf162b6e8d None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:15.670217 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2710de30-c86e-4bd6-a6e4-9ccf162b6e8d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:15.670832 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1ac632-f9f7-46ba-98d9-f83591ed4a0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:15.671457 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1ac632-f9f7-46ba-98d9-f83591ed4a0b None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:15.671736 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1ac632-f9f7-46ba-98d9-f83591ed4a0b None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:15.672010 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1ac632-f9f7-46ba-98d9-f83591ed4a0b None admin] PATH_INFO: `/v3/roles/service` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:15.681975 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2710de30-c86e-4bd6-a6e4-9ccf162b6e8d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:15.682664 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1ac632-f9f7-46ba-98d9-f83591ed4a0b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:17:15.683438 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2710de30-c86e-4bd6-a6e4-9ccf162b6e8d None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:15.687853 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-9a1ac632-f9f7-46ba-98d9-f83591ed4a0b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Nov 02 07:17:15.687853 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:17:15.687853 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:17:15.687853 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:17:15.687853 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:17:15.687853 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:17:15.687853 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:17:15.687853 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:17:15.687853 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:17:15.687853 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:17:15.687853 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:17:15.687853 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:17:15.687853 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:17:15.687853 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:17:15.687853 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:17:15.687853 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:17:15.688872 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:17:15.688872 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:17:15.688872 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:17:15.688872 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:17:15.688872 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:17:15.688872 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:17:15.688872 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:17:15.688872 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:17:15.688872 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:17:15.688872 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:17:15.688872 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:17:15.688872 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Nov 02 07:17:15.688872 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:17:15.688872 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:17:15.688872 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:17:15.688872 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:17:15.689691 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:17:15.689691 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:17:15.689691 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:17:15.689691 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:17:15.689691 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:17:15.689691 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:17:15.689691 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:17:15.689691 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:17:15.689691 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:17:15.689691 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:17:15.689691 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Nov 02 07:17:15.689691 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Nov 02 07:17:15.689691 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 220/447] 10.222.0.18 () {56 vars in 1102 bytes} [Wed Nov 2 07:17:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:17:15.690325 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1ac632-f9f7-46ba-98d9-f83591ed4a0b None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:15.699151 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-2710de30-c86e-4bd6-a6e4-9ccf162b6e8d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:17:15.699151 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:17:15.699151 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:17:15.699151 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:17:15.699151 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:17:15.699151 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:17:15.699151 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:17:15.699151 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:17:15.699151 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:17:15.699151 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:17:15.699151 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:17:15.699151 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:17:15.699151 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:17:15.699151 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:17:15.699151 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:17:15.699151 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:17:15.700191 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:17:15.700191 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:17:15.700191 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:17:15.700191 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:17:15.700191 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:17:15.700191 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:17:15.700191 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:17:15.700191 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:17:15.700191 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:17:15.700191 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:17:15.700191 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:17:15.700191 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:17:15.700191 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:17:15.700191 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:17:15.700191 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:17:15.700191 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:17:15.700191 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:17:15.701011 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:17:15.701011 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:17:15.701011 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:17:15.701011 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 228/448] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Nov 2 07:17:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:17:15.701011 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2710de30-c86e-4bd6-a6e4-9ccf162b6e8d None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:15.722687 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:15.793594 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1ac632-f9f7-46ba-98d9-f83591ed4a0b None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:15.797967 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2710de30-c86e-4bd6-a6e4-9ccf162b6e8d None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:15.826453 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7205849-1c0b-400b-a5cc-42181881c691 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:15.827075 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7205849-1c0b-400b-a5cc-42181881c691 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:15.827355 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7205849-1c0b-400b-a5cc-42181881c691 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:15.827625 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7205849-1c0b-400b-a5cc-42181881c691 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:15.830653 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a977b1-c0fc-417f-bc41-2f7f154b5587 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:15.831242 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a977b1-c0fc-417f-bc41-2f7f154b5587 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:15.831517 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a977b1-c0fc-417f-bc41-2f7f154b5587 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:15.831784 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a977b1-c0fc-417f-bc41-2f7f154b5587 None admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:15.832408 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a977b1-c0fc-417f-bc41-2f7f154b5587 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:17:15.832703 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a977b1-c0fc-417f-bc41-2f7f154b5587 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:15.834648 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a977b1-c0fc-417f-bc41-2f7f154b5587 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:15.834951 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7205849-1c0b-400b-a5cc-42181881c691 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:15.836414 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7205849-1c0b-400b-a5cc-42181881c691 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:15.845924 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 229/449] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Nov 2 07:17:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:17:15.846738 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a977b1-c0fc-417f-bc41-2f7f154b5587 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:15.847450 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-f7205849-1c0b-400b-a5cc-42181881c691 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 07:17:15.847450 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:17:15.847450 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:17:15.847450 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:17:15.847450 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:17:15.847450 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:17:15.847450 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:17:15.847450 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:17:15.847450 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:17:15.847450 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:17:15.847450 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:17:15.847450 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:17:15.847450 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Nov 02 07:17:15.847450 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_project(project_id) Nov 02 07:17:15.847450 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Nov 02 07:17:15.848324 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Nov 02 07:17:15.848324 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:17:15.848324 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:17:15.848324 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:17:15.848324 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:17:15.848324 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:17:15.848324 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:17:15.848324 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:17:15.848324 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Nov 02 07:17:15.848324 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:17:15.848324 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:17:15.848324 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:17:15.848324 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:17:15.848324 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:17:15.848324 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:17:15.848324 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:17:15.848324 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:17:15.849719 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Nov 02 07:17:15.849719 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Nov 02 07:17:15.849719 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Nov 02 07:17:15.849719 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Nov 02 07:17:15.849719 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Nov 02 07:17:15.849719 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Nov 02 07:17:15.849719 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 07:17:15.849719 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Nov 02 07:17:15.849719 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 221/450] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Nov 2 07:17:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:17:15.851808 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7205849-1c0b-400b-a5cc-42181881c691 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:15.878621 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:15.927945 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:15.947081 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a977b1-c0fc-417f-bc41-2f7f154b5587 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:15.952351 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7205849-1c0b-400b-a5cc-42181881c691 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:15.980799 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5de89c-e989-4a46-94cd-b5e92cf486dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:15.981416 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5de89c-e989-4a46-94cd-b5e92cf486dd None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:15.981690 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5de89c-e989-4a46-94cd-b5e92cf486dd None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:15.981955 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5de89c-e989-4a46-94cd-b5e92cf486dd None admin] PATH_INFO: `/v3/roles/service` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:15.985995 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62dee14-fc94-4e8b-99b6-ebb03c3d7f67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:15.986620 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62dee14-fc94-4e8b-99b6-ebb03c3d7f67 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:15.986894 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62dee14-fc94-4e8b-99b6-ebb03c3d7f67 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:15.987160 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62dee14-fc94-4e8b-99b6-ebb03c3d7f67 None admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:15.987806 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62dee14-fc94-4e8b-99b6-ebb03c3d7f67 None admin] RBAC: Adding query filter params (name=service) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:17:15.988101 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62dee14-fc94-4e8b-99b6-ebb03c3d7f67 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:15.990000 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62dee14-fc94-4e8b-99b6-ebb03c3d7f67 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:15.994002 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5de89c-e989-4a46-94cd-b5e92cf486dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:17:15.998668 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 222/451] 10.222.0.18 () {56 vars in 1103 bytes} [Wed Nov 2 07:17:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:17:15.999265 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-cd5de89c-e989-4a46-94cd-b5e92cf486dd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Nov 02 07:17:15.999265 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:17:15.999265 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:17:15.999265 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:17:15.999265 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:17:15.999265 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:17:15.999265 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:17:15.999265 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:17:15.999265 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:17:15.999265 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:17:15.999265 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:17:15.999265 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:17:15.999265 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:17:15.999265 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:17:15.999265 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:17:15.999265 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:17:16.000120 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:17:16.000120 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:17:16.000120 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:17:16.000120 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:17:16.000120 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:17:16.000120 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:17:16.000120 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:17:16.000120 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:17:16.000120 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:17:16.000120 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:17:16.000120 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:17:16.000120 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:17:16.000120 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:17:16.000120 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:17:16.000120 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:17:16.000120 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:17:16.000924 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:17:16.000924 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:17:16.000924 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:17:16.000924 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:17:16.000924 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:17:16.000924 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:17:16.000924 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:17:16.000924 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:17:16.000924 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:17:16.000924 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:17:16.000924 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Nov 02 07:17:16.000924 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:17:16.001529 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62dee14-fc94-4e8b-99b6-ebb03c3d7f67 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:16.001606 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 230/452] 10.222.0.18 () {56 vars in 1102 bytes} [Wed Nov 2 07:17:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:17:16.001606 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5de89c-e989-4a46-94cd-b5e92cf486dd None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:16.090514 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:16.099605 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5de89c-e989-4a46-94cd-b5e92cf486dd None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:16.100827 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62dee14-fc94-4e8b-99b6-ebb03c3d7f67 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:16.133205 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077f750d-ae1b-4efb-a34f-7f44d9d340c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:16.133441 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0b528c-4824-4018-a40d-1171a72f256c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:16.133816 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077f750d-ae1b-4efb-a34f-7f44d9d340c5 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:16.134038 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0b528c-4824-4018-a40d-1171a72f256c None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:16.134126 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077f750d-ae1b-4efb-a34f-7f44d9d340c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:16.134316 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0b528c-4824-4018-a40d-1171a72f256c None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:16.134436 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077f750d-ae1b-4efb-a34f-7f44d9d340c5 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:16.134616 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0b528c-4824-4018-a40d-1171a72f256c None admin] PATH_INFO: `/v3/domains` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:16.135245 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0b528c-4824-4018-a40d-1171a72f256c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:17:16.135543 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0b528c-4824-4018-a40d-1171a72f256c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:16.136913 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0b528c-4824-4018-a40d-1171a72f256c None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:16.141154 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077f750d-ae1b-4efb-a34f-7f44d9d340c5 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:16.142583 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077f750d-ae1b-4efb-a34f-7f44d9d340c5 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:16.144982 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:16.148846 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 223/453] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Nov 2 07:17:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:17:16.149648 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0b528c-4824-4018-a40d-1171a72f256c None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:16.158968 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-077f750d-ae1b-4efb-a34f-7f44d9d340c5 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Nov 02 07:17:16.158968 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:17:16.158968 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:17:16.158968 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:17:16.158968 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:17:16.158968 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:17:16.158968 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:17:16.158968 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:17:16.158968 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:17:16.158968 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:17:16.158968 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:17:16.158968 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:17:16.158968 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Nov 02 07:17:16.158968 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_user(user_id) Nov 02 07:17:16.158968 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Nov 02 07:17:16.158968 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Nov 02 07:17:16.159941 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:17:16.159941 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:17:16.159941 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Nov 02 07:17:16.159941 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:17:16.159941 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Nov 02 07:17:16.159941 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:17:16.159941 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:17:16.159941 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:17:16.159941 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:17:16.159941 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:17:16.159941 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:17:16.159941 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:17:16.159941 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:17:16.159941 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:17:16.159941 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:17:16.159941 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:17:16.159941 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:17:16.159941 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:17:16.160829 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:17:16.160829 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:17:16.160829 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Nov 02 07:17:16.160829 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Nov 02 07:17:16.160829 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Nov 02 07:17:16.160829 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Nov 02 07:17:16.160829 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Nov 02 07:17:16.160829 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Nov 02 07:17:16.160829 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Nov 02 07:17:16.160829 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:17:16.160829 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 231/454] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Nov 2 07:17:16 2022] GET /identity/v3/users/glance => generated 84 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:17:16.160829 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077f750d-ae1b-4efb-a34f-7f44d9d340c5 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:16.249756 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0b528c-4824-4018-a40d-1171a72f256c None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:16.259537 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077f750d-ae1b-4efb-a34f-7f44d9d340c5 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:16.282798 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f608ce-8ba2-4633-ab6e-ff5681a0ff49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:16.283391 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f608ce-8ba2-4633-ab6e-ff5681a0ff49 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:16.283673 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f608ce-8ba2-4633-ab6e-ff5681a0ff49 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:16.283957 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f608ce-8ba2-4633-ab6e-ff5681a0ff49 None admin] PATH_INFO: `/v3/projects` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:16.284628 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f608ce-8ba2-4633-ab6e-ff5681a0ff49 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:17:16.284924 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f608ce-8ba2-4633-ab6e-ff5681a0ff49 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:16.286861 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f608ce-8ba2-4633-ab6e-ff5681a0ff49 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:16.292902 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db50884-628c-4d9e-8b34-3e2de1b75033 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:16.293522 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db50884-628c-4d9e-8b34-3e2de1b75033 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:16.293792 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db50884-628c-4d9e-8b34-3e2de1b75033 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:16.294062 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db50884-628c-4d9e-8b34-3e2de1b75033 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:16.298099 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 224/455] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Nov 2 07:17:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:17:16.298734 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f608ce-8ba2-4633-ab6e-ff5681a0ff49 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:16.307395 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db50884-628c-4d9e-8b34-3e2de1b75033 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:16.308878 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db50884-628c-4d9e-8b34-3e2de1b75033 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:16.314022 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:16.323782 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-1db50884-628c-4d9e-8b34-3e2de1b75033 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:17:16.323782 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:17:16.323782 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:17:16.323782 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:17:16.323782 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:17:16.323782 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:17:16.323782 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:17:16.323782 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:17:16.323782 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:17:16.323782 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:17:16.323782 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:17:16.323782 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:17:16.323782 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:17:16.323782 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:17:16.323782 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:17:16.323782 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:17:16.324788 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:17:16.324788 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:17:16.324788 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:17:16.324788 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:17:16.324788 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:17:16.324788 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:17:16.324788 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:17:16.324788 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:17:16.324788 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:17:16.324788 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:17:16.324788 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:17:16.324788 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:17:16.324788 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:17:16.324788 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:17:16.324788 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:17:16.324788 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:17:16.324788 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:17:16.325650 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:17:16.325650 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:17:16.325650 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:17:16.325650 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 232/456] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Nov 2 07:17:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:17:16.325861 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db50884-628c-4d9e-8b34-3e2de1b75033 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:16.379276 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:16.405708 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f608ce-8ba2-4633-ab6e-ff5681a0ff49 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:16.425152 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db50884-628c-4d9e-8b34-3e2de1b75033 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:16.439512 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48474f45-33cd-4c53-b0a3-f1994aa603c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:16.440115 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48474f45-33cd-4c53-b0a3-f1994aa603c5 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:16.440389 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48474f45-33cd-4c53-b0a3-f1994aa603c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:16.440659 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48474f45-33cd-4c53-b0a3-f1994aa603c5 None admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:16.441311 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48474f45-33cd-4c53-b0a3-f1994aa603c5 None admin] RBAC: Adding query filter params (name=service) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:17:16.441611 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48474f45-33cd-4c53-b0a3-f1994aa603c5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:16.443075 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48474f45-33cd-4c53-b0a3-f1994aa603c5 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:16.450849 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 225/457] 10.222.0.18 () {56 vars in 1103 bytes} [Wed Nov 2 07:17:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:17:16.451620 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48474f45-33cd-4c53-b0a3-f1994aa603c5 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:16.459936 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dff63b-af24-4c50-8c17-5f78883b6fa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:16.460558 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dff63b-af24-4c50-8c17-5f78883b6fa5 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:16.460838 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dff63b-af24-4c50-8c17-5f78883b6fa5 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:16.461105 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dff63b-af24-4c50-8c17-5f78883b6fa5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:16.467778 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23dff63b-af24-4c50-8c17-5f78883b6fa5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:16.469185 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23dff63b-af24-4c50-8c17-5f78883b6fa5 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:16.481693 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-23dff63b-af24-4c50-8c17-5f78883b6fa5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 07:17:16.481693 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:17:16.481693 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:17:16.481693 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:17:16.481693 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:17:16.481693 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:17:16.481693 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:17:16.481693 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:17:16.481693 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:17:16.481693 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:17:16.481693 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:17:16.481693 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:17:16.481693 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Nov 02 07:17:16.481693 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_project(project_id) Nov 02 07:17:16.481693 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Nov 02 07:17:16.482807 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Nov 02 07:17:16.482807 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:17:16.482807 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:17:16.482807 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:17:16.482807 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:17:16.482807 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:17:16.482807 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:17:16.482807 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:17:16.482807 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:17:16.482807 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:17:16.482807 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:17:16.482807 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:17:16.482807 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:17:16.482807 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:17:16.482807 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:17:16.482807 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:17:16.482807 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:17:16.483644 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Nov 02 07:17:16.483644 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Nov 02 07:17:16.483644 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Nov 02 07:17:16.483644 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Nov 02 07:17:16.483644 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Nov 02 07:17:16.483644 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Nov 02 07:17:16.483644 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 07:17:16.483644 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:17:16.483644 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 233/458] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Nov 2 07:17:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:17:16.483644 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dff63b-af24-4c50-8c17-5f78883b6fa5 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:16.523096 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:16.553105 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48474f45-33cd-4c53-b0a3-f1994aa603c5 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:16.580294 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dff63b-af24-4c50-8c17-5f78883b6fa5 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:16.585797 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc841e1-de76-4a73-aafe-43a6153043cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:16.586429 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc841e1-de76-4a73-aafe-43a6153043cc None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:16.586714 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc841e1-de76-4a73-aafe-43a6153043cc None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:16.586984 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc841e1-de76-4a73-aafe-43a6153043cc None admin] PATH_INFO: `/v3/users` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:16.587697 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc841e1-de76-4a73-aafe-43a6153043cc None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:17:16.588001 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc841e1-de76-4a73-aafe-43a6153043cc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:16.589425 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc841e1-de76-4a73-aafe-43a6153043cc None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:16.593376 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:16.607312 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 226/459] 10.222.0.18 () {56 vars in 1137 bytes} [Wed Nov 2 07:17:16 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:17:16.608119 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc841e1-de76-4a73-aafe-43a6153043cc None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:16.619750 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f31c10-55d0-4519-a1ae-239b857f9c87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:16.620348 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f31c10-55d0-4519-a1ae-239b857f9c87 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:16.620624 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f31c10-55d0-4519-a1ae-239b857f9c87 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:16.620891 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f31c10-55d0-4519-a1ae-239b857f9c87 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:16.621599 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f31c10-55d0-4519-a1ae-239b857f9c87 None admin] RBAC: Adding query filter params (role.id=4af6f4c514284b8cbddf3e02ec826378, scope.project.id=af04daa468b64160a6176dcab29502a7, user.id=69e48cba3f804cf08b2477f01ae95179) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:17:16.621931 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f31c10-55d0-4519-a1ae-239b857f9c87 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:16.623382 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f31c10-55d0-4519-a1ae-239b857f9c87 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:16.629040 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 234/460] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Nov 2 07:17:16 2022] GET /identity/v3/role_assignments?user.id=69e48cba3f804cf08b2477f01ae95179&scope.project.id=af04daa468b64160a6176dcab29502a7&role.id=4af6f4c514284b8cbddf3e02ec826378 => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:17:16.629803 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f31c10-55d0-4519-a1ae-239b857f9c87 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:16.730110 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc841e1-de76-4a73-aafe-43a6153043cc None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:16.759887 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f31c10-55d0-4519-a1ae-239b857f9c87 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:16.762716 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:16.771755 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18811495-eaeb-4643-b163-5af18e6cbd52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:16.772380 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18811495-eaeb-4643-b163-5af18e6cbd52 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:16.772659 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18811495-eaeb-4643-b163-5af18e6cbd52 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:16.772927 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18811495-eaeb-4643-b163-5af18e6cbd52 None admin] PATH_INFO: `/v3/domains` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:16.774173 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18811495-eaeb-4643-b163-5af18e6cbd52 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:17:16.774462 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18811495-eaeb-4643-b163-5af18e6cbd52 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:16.776166 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18811495-eaeb-4643-b163-5af18e6cbd52 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:16.793386 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 227/461] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Nov 2 07:17:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:17:16.793386 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18811495-eaeb-4643-b163-5af18e6cbd52 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:16.806225 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3322563-df34-43b5-8a6a-aea810272cb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:16.806887 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3322563-df34-43b5-8a6a-aea810272cb1 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:16.807252 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3322563-df34-43b5-8a6a-aea810272cb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:16.807444 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3322563-df34-43b5-8a6a-aea810272cb1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:16.819787 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3322563-df34-43b5-8a6a-aea810272cb1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:16.821225 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3322563-df34-43b5-8a6a-aea810272cb1 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:16.838339 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-d3322563-df34-43b5-8a6a-aea810272cb1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:17:16.838339 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:17:16.838339 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:17:16.838339 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:17:16.838339 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:17:16.838339 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:17:16.838339 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:17:16.838339 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:17:16.838339 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:17:16.838339 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:17:16.838339 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:17:16.838339 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:17:16.838339 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:17:16.838339 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:17:16.838339 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:17:16.838339 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:17:16.839269 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:17:16.839269 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:17:16.839269 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:17:16.839269 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:17:16.839269 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:17:16.839269 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:17:16.839269 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:17:16.839269 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:17:16.839269 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:17:16.839269 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:17:16.839269 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:17:16.839269 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:17:16.839269 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:17:16.839269 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:17:16.839269 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:17:16.839269 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:17:16.839269 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:17:16.840108 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:17:16.840108 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:17:16.840108 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:17:16.844012 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 235/462] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Nov 2 07:17:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:17:16.844845 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3322563-df34-43b5-8a6a-aea810272cb1 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:16.877715 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:16.909092 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18811495-eaeb-4643-b163-5af18e6cbd52 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:16.948130 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d04d20f-3110-4bb4-913b-de00f99c82a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:16.949080 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d04d20f-3110-4bb4-913b-de00f99c82a9 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:16.949529 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d04d20f-3110-4bb4-913b-de00f99c82a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:16.949972 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d04d20f-3110-4bb4-913b-de00f99c82a9 None admin] PATH_INFO: `/v3/projects` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:16.951148 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d04d20f-3110-4bb4-913b-de00f99c82a9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:17:16.951606 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d04d20f-3110-4bb4-913b-de00f99c82a9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:16.954065 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d04d20f-3110-4bb4-913b-de00f99c82a9 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:16.969682 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3322563-df34-43b5-8a6a-aea810272cb1 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:16.973759 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 228/463] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Nov 2 07:17:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:17:16.974874 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d04d20f-3110-4bb4-913b-de00f99c82a9 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:17.018251 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cc6915-dc39-4609-b431-ce2b3373f531 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:17.018913 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cc6915-dc39-4609-b431-ce2b3373f531 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:17.019199 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cc6915-dc39-4609-b431-ce2b3373f531 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:17.019503 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cc6915-dc39-4609-b431-ce2b3373f531 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:17.030919 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:17.087667 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d04d20f-3110-4bb4-913b-de00f99c82a9 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:17.092870 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cc6915-dc39-4609-b431-ce2b3373f531 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:17.094331 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cc6915-dc39-4609-b431-ce2b3373f531 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:17.123385 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:17.124671 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec206bb-26b8-4a02-b070-c2b2f512f0f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:17.125308 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec206bb-26b8-4a02-b070-c2b2f512f0f1 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:17.125588 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec206bb-26b8-4a02-b070-c2b2f512f0f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:17.125946 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec206bb-26b8-4a02-b070-c2b2f512f0f1 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:17.135072 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec206bb-26b8-4a02-b070-c2b2f512f0f1 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:17.137146 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec206bb-26b8-4a02-b070-c2b2f512f0f1 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:17.150133 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-eec206bb-26b8-4a02-b070-c2b2f512f0f1 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Nov 02 07:17:17.150133 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:17:17.150133 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:17:17.150133 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:17:17.150133 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:17:17.150133 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:17:17.150133 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:17:17.150133 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:17:17.150133 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:17:17.150133 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:17:17.150133 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:17:17.150133 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:17:17.150133 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Nov 02 07:17:17.150133 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_user(user_id) Nov 02 07:17:17.150133 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Nov 02 07:17:17.150133 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Nov 02 07:17:17.151223 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:17:17.151223 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:17:17.151223 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Nov 02 07:17:17.151223 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:17:17.151223 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Nov 02 07:17:17.151223 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:17:17.151223 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:17:17.151223 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:17:17.151223 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:17:17.151223 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:17:17.151223 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:17:17.151223 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Nov 02 07:17:17.151223 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:17:17.151223 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:17:17.151223 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:17:17.151223 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:17:17.151223 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:17:17.151223 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:17:17.152122 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:17:17.152122 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:17:17.152122 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Nov 02 07:17:17.152122 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Nov 02 07:17:17.152122 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Nov 02 07:17:17.152122 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Nov 02 07:17:17.152122 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Nov 02 07:17:17.152122 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Nov 02 07:17:17.152122 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Nov 02 07:17:17.152122 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Nov 02 07:17:17.152122 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 229/464] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Nov 2 07:17:16 2022] GET /identity/v3/users/cinder => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:17:17.152666 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec206bb-26b8-4a02-b070-c2b2f512f0f1 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:17.209079 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 236/465] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Nov 2 07:17:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Nov 02 07:17:17.209818 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cc6915-dc39-4609-b431-ce2b3373f531 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:17.265377 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec206bb-26b8-4a02-b070-c2b2f512f0f1 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:17.277069 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:17.304214 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22069796-72e7-4bb2-bed2-ad22b74fbd94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:17.304737 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22069796-72e7-4bb2-bed2-ad22b74fbd94 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:17.305018 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22069796-72e7-4bb2-bed2-ad22b74fbd94 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:17.305278 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22069796-72e7-4bb2-bed2-ad22b74fbd94 None admin] PATH_INFO: `/v3/domains` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:17.305909 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22069796-72e7-4bb2-bed2-ad22b74fbd94 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:17:17.306204 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22069796-72e7-4bb2-bed2-ad22b74fbd94 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:17.307588 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22069796-72e7-4bb2-bed2-ad22b74fbd94 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:17.315002 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cc6915-dc39-4609-b431-ce2b3373f531 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:17.325351 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 230/466] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Nov 2 07:17:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:17:17.325351 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22069796-72e7-4bb2-bed2-ad22b74fbd94 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:17.344479 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:17.353412 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be283dd6-c855-4950-9685-ed3cb7071895 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:17.354061 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be283dd6-c855-4950-9685-ed3cb7071895 None admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:17.354336 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be283dd6-c855-4950-9685-ed3cb7071895 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:17.354622 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be283dd6-c855-4950-9685-ed3cb7071895 None admin] PATH_INFO: `/v3/projects/af04daa468b64160a6176dcab29502a7/users/f9116f47889f4db39e0203d6cdae1593/roles/4af6f4c514284b8cbddf3e02ec826378` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:17.370171 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be283dd6-c855-4950-9685-ed3cb7071895 None admin] RBAC: Authorizing `identity:create_grant(project_id=af04daa468b64160a6176dcab29502a7, user_id=f9116f47889f4db39e0203d6cdae1593, role_id=4af6f4c514284b8cbddf3e02ec826378)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:17.371636 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be283dd6-c855-4950-9685-ed3cb7071895 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:17.386688 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 237/467] 10.222.0.18 () {58 vars in 1439 bytes} [Wed Nov 2 07:17:17 2022] PUT /identity/v3/projects/af04daa468b64160a6176dcab29502a7/users/f9116f47889f4db39e0203d6cdae1593/roles/4af6f4c514284b8cbddf3e02ec826378 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:17:17.387815 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be283dd6-c855-4950-9685-ed3cb7071895 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:17.457129 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22069796-72e7-4bb2-bed2-ad22b74fbd94 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:17.511835 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b826965-cbab-4e59-888b-0ec894ed48da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:17.513004 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b826965-cbab-4e59-888b-0ec894ed48da None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:17.513302 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b826965-cbab-4e59-888b-0ec894ed48da None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:17.513572 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b826965-cbab-4e59-888b-0ec894ed48da None admin] PATH_INFO: `/v3/users` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:17.514631 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b826965-cbab-4e59-888b-0ec894ed48da None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:17:17.514941 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b826965-cbab-4e59-888b-0ec894ed48da None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:17.516721 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b826965-cbab-4e59-888b-0ec894ed48da None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:17.524163 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be283dd6-c855-4950-9685-ed3cb7071895 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:17.546090 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 231/468] 10.222.0.18 () {56 vars in 1137 bytes} [Wed Nov 2 07:17:17 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:17:17.547506 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b826965-cbab-4e59-888b-0ec894ed48da None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:17.572859 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba844a02-e2d2-48f5-bcf6-1b22ab564720 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:17.573418 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba844a02-e2d2-48f5-bcf6-1b22ab564720 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:17.573697 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba844a02-e2d2-48f5-bcf6-1b22ab564720 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:17.573965 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba844a02-e2d2-48f5-bcf6-1b22ab564720 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:17.587216 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba844a02-e2d2-48f5-bcf6-1b22ab564720 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:17.587653 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:17.588716 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba844a02-e2d2-48f5-bcf6-1b22ab564720 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:17.607252 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-ba844a02-e2d2-48f5-bcf6-1b22ab564720 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:17:17.607252 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:17:17.607252 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:17:17.607252 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:17:17.607252 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:17:17.607252 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:17:17.607252 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:17:17.607252 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:17:17.607252 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:17:17.607252 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:17:17.607252 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:17:17.607252 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:17:17.607252 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:17:17.607252 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:17:17.607252 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:17:17.607252 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:17:17.608184 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:17:17.608184 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:17:17.608184 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:17:17.608184 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:17:17.608184 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:17:17.608184 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:17:17.608184 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:17:17.608184 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:17:17.608184 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:17:17.608184 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:17:17.608184 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:17:17.608184 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:17:17.608184 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:17:17.608184 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:17:17.608184 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:17:17.608184 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:17:17.608184 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:17:17.609028 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:17:17.609028 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:17:17.609028 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:17:17.609028 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 238/469] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Nov 2 07:17:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:17:17.609701 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba844a02-e2d2-48f5-bcf6-1b22ab564720 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:17.652474 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:17.687432 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b826965-cbab-4e59-888b-0ec894ed48da None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:17.726923 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c29de93-a449-483e-a4f0-431bc8f6f2f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:17.727893 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c29de93-a449-483e-a4f0-431bc8f6f2f2 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:17.728238 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c29de93-a449-483e-a4f0-431bc8f6f2f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:17.728639 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c29de93-a449-483e-a4f0-431bc8f6f2f2 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:17.735943 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c29de93-a449-483e-a4f0-431bc8f6f2f2 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:17.737489 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c29de93-a449-483e-a4f0-431bc8f6f2f2 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:17.741158 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba844a02-e2d2-48f5-bcf6-1b22ab564720 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:17.749585 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-2c29de93-a449-483e-a4f0-431bc8f6f2f2 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Nov 02 07:17:17.749585 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:17:17.749585 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:17:17.749585 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:17:17.749585 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:17:17.749585 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:17:17.749585 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:17:17.749585 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:17:17.749585 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:17:17.749585 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:17:17.749585 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:17:17.749585 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:17:17.749585 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Nov 02 07:17:17.749585 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_user(user_id) Nov 02 07:17:17.749585 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Nov 02 07:17:17.749585 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Nov 02 07:17:17.750507 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:17:17.750507 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:17:17.750507 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Nov 02 07:17:17.750507 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:17:17.750507 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Nov 02 07:17:17.750507 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:17:17.750507 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:17:17.750507 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:17:17.750507 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:17:17.750507 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:17:17.750507 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:17:17.750507 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Nov 02 07:17:17.750507 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:17:17.750507 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:17:17.750507 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:17:17.750507 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:17:17.750507 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:17:17.750507 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:17:17.751397 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:17:17.751397 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:17:17.751397 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Nov 02 07:17:17.751397 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Nov 02 07:17:17.751397 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Nov 02 07:17:17.751397 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Nov 02 07:17:17.751397 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Nov 02 07:17:17.751397 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Nov 02 07:17:17.751397 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Nov 02 07:17:17.751397 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Nov 02 07:17:17.751397 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 232/470] 10.222.0.18 () {56 vars in 1102 bytes} [Wed Nov 2 07:17:17 2022] GET /identity/v3/users/neutron => generated 85 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:17:17.751948 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c29de93-a449-483e-a4f0-431bc8f6f2f2 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:17.794677 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41176893-769f-4f60-8994-f5ed6d0afdf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:17.795828 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41176893-769f-4f60-8994-f5ed6d0afdf2 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:17.797141 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41176893-769f-4f60-8994-f5ed6d0afdf2 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:17.797672 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41176893-769f-4f60-8994-f5ed6d0afdf2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:17.853768 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:17.867179 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c29de93-a449-483e-a4f0-431bc8f6f2f2 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:17.877530 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41176893-769f-4f60-8994-f5ed6d0afdf2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:17.880742 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41176893-769f-4f60-8994-f5ed6d0afdf2 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:17.904256 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7755164-9ded-4717-a099-d66b92eb2cb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:17.904862 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7755164-9ded-4717-a099-d66b92eb2cb6 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:17.905137 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7755164-9ded-4717-a099-d66b92eb2cb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:17.905411 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7755164-9ded-4717-a099-d66b92eb2cb6 None admin] PATH_INFO: `/v3/domains` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:17.906051 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7755164-9ded-4717-a099-d66b92eb2cb6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:17:17.906450 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7755164-9ded-4717-a099-d66b92eb2cb6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:17.907811 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7755164-9ded-4717-a099-d66b92eb2cb6 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:17.920419 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 233/471] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Nov 2 07:17:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:17:17.921101 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7755164-9ded-4717-a099-d66b92eb2cb6 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:17.923616 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:18.016790 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 239/472] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Nov 2 07:17:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Nov 02 07:17:18.017670 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41176893-769f-4f60-8994-f5ed6d0afdf2 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:18.049271 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7755164-9ded-4717-a099-d66b92eb2cb6 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:18.085373 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95162e2-b2c5-4b84-86c2-7f643b3f0743 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:18.085971 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95162e2-b2c5-4b84-86c2-7f643b3f0743 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:18.086242 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95162e2-b2c5-4b84-86c2-7f643b3f0743 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:18.086554 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95162e2-b2c5-4b84-86c2-7f643b3f0743 None admin] PATH_INFO: `/v3/users` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:18.087312 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95162e2-b2c5-4b84-86c2-7f643b3f0743 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:17:18.087620 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95162e2-b2c5-4b84-86c2-7f643b3f0743 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:18.089034 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95162e2-b2c5-4b84-86c2-7f643b3f0743 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:18.112166 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 234/473] 10.222.0.18 () {56 vars in 1139 bytes} [Wed Nov 2 07:17:17 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:17:18.112166 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95162e2-b2c5-4b84-86c2-7f643b3f0743 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:18.124130 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:18.148451 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41176893-769f-4f60-8994-f5ed6d0afdf2 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:18.180876 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:18.187296 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbf2c79-bfe1-43da-891d-5ad7f6f5ef43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:18.188178 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbf2c79-bfe1-43da-891d-5ad7f6f5ef43 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:18.188676 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbf2c79-bfe1-43da-891d-5ad7f6f5ef43 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:18.189206 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbf2c79-bfe1-43da-891d-5ad7f6f5ef43 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:18.197190 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbf2c79-bfe1-43da-891d-5ad7f6f5ef43 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:18.199018 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbf2c79-bfe1-43da-891d-5ad7f6f5ef43 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:18.212106 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-abbf2c79-bfe1-43da-891d-5ad7f6f5ef43 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 07:17:18.212106 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:17:18.212106 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:17:18.212106 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:17:18.212106 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:17:18.212106 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:17:18.212106 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:17:18.212106 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:17:18.212106 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:17:18.212106 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:17:18.212106 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:17:18.212106 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:17:18.212106 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Nov 02 07:17:18.212106 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_project(project_id) Nov 02 07:17:18.212106 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Nov 02 07:17:18.212956 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Nov 02 07:17:18.212956 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:17:18.212956 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:17:18.212956 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:17:18.212956 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:17:18.212956 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:17:18.212956 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:17:18.212956 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:17:18.212956 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:17:18.212956 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:17:18.212956 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:17:18.212956 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:17:18.212956 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:17:18.212956 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:17:18.212956 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:17:18.212956 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:17:18.212956 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:17:18.213801 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Nov 02 07:17:18.213801 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Nov 02 07:17:18.213801 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Nov 02 07:17:18.213801 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Nov 02 07:17:18.213801 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Nov 02 07:17:18.213801 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Nov 02 07:17:18.213801 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 07:17:18.213801 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:17:18.215988 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 240/474] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Nov 2 07:17:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:17:18.217086 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbf2c79-bfe1-43da-891d-5ad7f6f5ef43 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:18.225845 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95162e2-b2c5-4b84-86c2-7f643b3f0743 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:18.261715 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f98ba3-67ae-47ec-971b-c4297407ecb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:18.262669 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f98ba3-67ae-47ec-971b-c4297407ecb4 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:18.262669 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f98ba3-67ae-47ec-971b-c4297407ecb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:18.262953 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f98ba3-67ae-47ec-971b-c4297407ecb4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:18.273594 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f98ba3-67ae-47ec-971b-c4297407ecb4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:18.275070 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f98ba3-67ae-47ec-971b-c4297407ecb4 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:18.290546 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-b1f98ba3-67ae-47ec-971b-c4297407ecb4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:17:18.290546 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:17:18.290546 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:17:18.290546 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:17:18.290546 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:17:18.290546 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:17:18.290546 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:17:18.290546 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:17:18.290546 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:17:18.290546 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:17:18.290546 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:17:18.290546 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:17:18.290546 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:17:18.290546 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:17:18.290546 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:17:18.290546 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:17:18.291469 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:17:18.291469 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:17:18.291469 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:17:18.291469 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:17:18.291469 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:17:18.291469 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:17:18.291469 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:17:18.291469 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Nov 02 07:17:18.291469 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:17:18.291469 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:17:18.291469 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:17:18.291469 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:17:18.291469 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:17:18.291469 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:17:18.291469 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:17:18.291469 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:17:18.291469 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:17:18.292348 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:17:18.292348 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:17:18.292348 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Nov 02 07:17:18.296509 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 235/475] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Nov 2 07:17:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:17:18.297362 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f98ba3-67ae-47ec-971b-c4297407ecb4 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:18.336235 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbf2c79-bfe1-43da-891d-5ad7f6f5ef43 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:18.363805 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:18.375662 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7dfe2fc-d5f7-428b-a634-28e10d339ef7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:18.376563 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dfe2fc-d5f7-428b-a634-28e10d339ef7 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:18.377037 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dfe2fc-d5f7-428b-a634-28e10d339ef7 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:18.377521 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dfe2fc-d5f7-428b-a634-28e10d339ef7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:18.408679 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f98ba3-67ae-47ec-971b-c4297407ecb4 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:18.441290 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:18.447762 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bbd111-ac64-4030-b7e6-f72481df861a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:18.448370 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bbd111-ac64-4030-b7e6-f72481df861a None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:18.448760 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bbd111-ac64-4030-b7e6-f72481df861a None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:18.448931 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bbd111-ac64-4030-b7e6-f72481df861a None admin] PATH_INFO: `/v3/projects` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:18.449585 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bbd111-ac64-4030-b7e6-f72481df861a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:17:18.449878 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bbd111-ac64-4030-b7e6-f72481df861a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:18.451384 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bbd111-ac64-4030-b7e6-f72481df861a None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:18.455613 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7dfe2fc-d5f7-428b-a634-28e10d339ef7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:18.457288 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7dfe2fc-d5f7-428b-a634-28e10d339ef7 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:18.464504 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 236/476] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Nov 2 07:17:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:17:18.465480 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bbd111-ac64-4030-b7e6-f72481df861a None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:18.566083 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 241/477] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Nov 2 07:17:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Nov 02 07:17:18.571459 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7dfe2fc-d5f7-428b-a634-28e10d339ef7 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:18.572373 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bbd111-ac64-4030-b7e6-f72481df861a None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:18.591486 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:18.611134 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f975216a-173a-4f8f-bc54-7f1f31a46fcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:18.611742 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f975216a-173a-4f8f-bc54-7f1f31a46fcc None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:18.612068 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f975216a-173a-4f8f-bc54-7f1f31a46fcc None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:18.612341 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f975216a-173a-4f8f-bc54-7f1f31a46fcc None admin] PATH_INFO: `/v3/domains` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:18.613582 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f975216a-173a-4f8f-bc54-7f1f31a46fcc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:17:18.613582 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f975216a-173a-4f8f-bc54-7f1f31a46fcc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:18.614883 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f975216a-173a-4f8f-bc54-7f1f31a46fcc None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:18.627240 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 237/478] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Nov 2 07:17:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:17:18.628035 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f975216a-173a-4f8f-bc54-7f1f31a46fcc None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:18.683451 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:18.694017 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7dfe2fc-d5f7-428b-a634-28e10d339ef7 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:18.732291 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8874a579-15cc-47e2-abd1-498ed6d1cbd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:18.733333 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8874a579-15cc-47e2-abd1-498ed6d1cbd4 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:18.733811 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8874a579-15cc-47e2-abd1-498ed6d1cbd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:18.734339 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8874a579-15cc-47e2-abd1-498ed6d1cbd4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:18.735480 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8874a579-15cc-47e2-abd1-498ed6d1cbd4 None admin] RBAC: Adding query filter params (role.id=4af6f4c514284b8cbddf3e02ec826378, scope.project.id=af04daa468b64160a6176dcab29502a7, user.id=c5353f2af0f44d668e2a269647aa1dc2) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:17:18.736214 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8874a579-15cc-47e2-abd1-498ed6d1cbd4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:18.737925 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8874a579-15cc-47e2-abd1-498ed6d1cbd4 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:18.746041 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 242/479] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Nov 2 07:17:18 2022] GET /identity/v3/role_assignments?user.id=c5353f2af0f44d668e2a269647aa1dc2&scope.project.id=af04daa468b64160a6176dcab29502a7&role.id=4af6f4c514284b8cbddf3e02ec826378 => generated 608 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:17:18.746234 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f975216a-173a-4f8f-bc54-7f1f31a46fcc None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:18.747465 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8874a579-15cc-47e2-abd1-498ed6d1cbd4 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:18.786703 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae63dd85-d0ca-42c4-9993-4f52f0b2df6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:18.787288 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae63dd85-d0ca-42c4-9993-4f52f0b2df6b None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:18.787565 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae63dd85-d0ca-42c4-9993-4f52f0b2df6b None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:18.787832 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae63dd85-d0ca-42c4-9993-4f52f0b2df6b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:18.800100 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae63dd85-d0ca-42c4-9993-4f52f0b2df6b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:18.801705 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae63dd85-d0ca-42c4-9993-4f52f0b2df6b None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:18.819045 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-ae63dd85-d0ca-42c4-9993-4f52f0b2df6b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:17:18.819045 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:17:18.819045 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:17:18.819045 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:17:18.819045 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:17:18.819045 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:17:18.819045 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:17:18.819045 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:17:18.819045 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:17:18.819045 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:17:18.819045 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:17:18.819045 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:17:18.819045 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:17:18.819045 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:17:18.819045 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:17:18.819045 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:17:18.819999 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:17:18.819999 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:17:18.819999 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:17:18.819999 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:17:18.819999 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:17:18.819999 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:17:18.819999 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:17:18.819999 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Nov 02 07:17:18.819999 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:17:18.819999 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:17:18.819999 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:17:18.819999 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:17:18.819999 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:17:18.819999 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:17:18.819999 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:17:18.819999 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:17:18.819999 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:17:18.820973 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:17:18.820973 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:17:18.820973 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Nov 02 07:17:18.820973 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 238/480] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Nov 2 07:17:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:17:18.826230 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae63dd85-d0ca-42c4-9993-4f52f0b2df6b None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:18.866490 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:18.877873 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8874a579-15cc-47e2-abd1-498ed6d1cbd4 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:18.923104 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7380211-eb0f-48ef-88f5-9726909bfb5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:18.923753 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7380211-eb0f-48ef-88f5-9726909bfb5e None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:18.924029 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7380211-eb0f-48ef-88f5-9726909bfb5e None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:18.924292 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7380211-eb0f-48ef-88f5-9726909bfb5e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:18.932584 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7380211-eb0f-48ef-88f5-9726909bfb5e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:18.934262 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7380211-eb0f-48ef-88f5-9726909bfb5e None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:18.947481 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-c7380211-eb0f-48ef-88f5-9726909bfb5e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 07:17:18.947481 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:17:18.947481 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:17:18.947481 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:17:18.947481 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:17:18.947481 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:17:18.947481 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:17:18.947481 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:17:18.947481 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:17:18.947481 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:17:18.947481 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:17:18.947481 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:17:18.947481 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Nov 02 07:17:18.947481 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_project(project_id) Nov 02 07:17:18.947481 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Nov 02 07:17:18.948579 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Nov 02 07:17:18.948579 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:17:18.948579 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:17:18.948579 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:17:18.948579 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:17:18.948579 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:17:18.948579 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:17:18.948579 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:17:18.948579 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:17:18.948579 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:17:18.948579 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:17:18.948579 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:17:18.948579 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:17:18.948579 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:17:18.948579 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:17:18.948579 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:17:18.948579 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:17:18.949579 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Nov 02 07:17:18.949579 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Nov 02 07:17:18.949579 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Nov 02 07:17:18.949579 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Nov 02 07:17:18.949579 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Nov 02 07:17:18.949579 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Nov 02 07:17:18.949579 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 07:17:18.949579 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:17:18.949579 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 243/481] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Nov 2 07:17:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:17:18.950025 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888b8a22-13ae-41d5-9d79-0e56b54eef4f None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:18.950088 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888b8a22-13ae-41d5-9d79-0e56b54eef4f None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:18.950377 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888b8a22-13ae-41d5-9d79-0e56b54eef4f None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:18.952846 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae63dd85-d0ca-42c4-9993-4f52f0b2df6b None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:18.956078 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 244/482] 10.222.0.18 () {54 vars in 927 bytes} [Wed Nov 2 07:17:18 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:17:18.957318 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888b8a22-13ae-41d5-9d79-0e56b54eef4f None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:18.976174 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:18.998350 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5a63be-5d4b-473e-9d44-d95cd945953a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:18.998980 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5a63be-5d4b-473e-9d44-d95cd945953a None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:18.999257 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5a63be-5d4b-473e-9d44-d95cd945953a None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:18.999523 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5a63be-5d4b-473e-9d44-d95cd945953a None admin] PATH_INFO: `/v3/domains` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:19.000159 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5a63be-5d4b-473e-9d44-d95cd945953a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:17:19.001367 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5a63be-5d4b-473e-9d44-d95cd945953a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:19.001827 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5a63be-5d4b-473e-9d44-d95cd945953a None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:19.024494 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 239/483] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Nov 2 07:17:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:17:19.025737 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31be829-5abe-4642-a2aa-6c6f6c5d2186 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:19.026022 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31be829-5abe-4642-a2aa-6c6f6c5d2186 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:19.026293 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31be829-5abe-4642-a2aa-6c6f6c5d2186 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:19.049623 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-e31be829-5abe-4642-a2aa-6c6f6c5d2186 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:17:19.050911 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-e31be829-5abe-4642-a2aa-6c6f6c5d2186 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:17:19.091825 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888b8a22-13ae-41d5-9d79-0e56b54eef4f None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:19.127955 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 240/484] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Nov 2 07:17:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:17:19.129200 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31be829-5abe-4642-a2aa-6c6f6c5d2186 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:19.140931 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf15c95-0f39-41de-b782-8c2cfd97248f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:19.142088 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf15c95-0f39-41de-b782-8c2cfd97248f None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:19.142473 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf15c95-0f39-41de-b782-8c2cfd97248f None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:19.142907 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf15c95-0f39-41de-b782-8c2cfd97248f None admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:19.144073 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf15c95-0f39-41de-b782-8c2cfd97248f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:17:19.144447 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf15c95-0f39-41de-b782-8c2cfd97248f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:19.147080 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf15c95-0f39-41de-b782-8c2cfd97248f None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:19.163417 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:19.173208 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 245/485] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Nov 2 07:17:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:17:19.178143 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf15c95-0f39-41de-b782-8c2cfd97248f None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:19.295527 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31be829-5abe-4642-a2aa-6c6f6c5d2186 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:19.318189 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:19.362949 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4806857-d510-4a2c-b735-bd75e091d584 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:19.365449 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4806857-d510-4a2c-b735-bd75e091d584 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:19.366136 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf15c95-0f39-41de-b782-8c2cfd97248f None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:19.366642 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4806857-d510-4a2c-b735-bd75e091d584 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:19.368504 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4806857-d510-4a2c-b735-bd75e091d584 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:19.381797 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4806857-d510-4a2c-b735-bd75e091d584 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:19.383528 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4806857-d510-4a2c-b735-bd75e091d584 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:19.406684 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-a4806857-d510-4a2c-b735-bd75e091d584 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 07:17:19.406684 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:17:19.406684 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:17:19.406684 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:17:19.406684 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:17:19.406684 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:17:19.406684 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:17:19.406684 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:17:19.406684 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:17:19.406684 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:17:19.406684 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:17:19.406684 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:17:19.406684 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Nov 02 07:17:19.406684 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_project(project_id) Nov 02 07:17:19.406684 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Nov 02 07:17:19.407648 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Nov 02 07:17:19.407648 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:17:19.407648 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:17:19.407648 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:17:19.407648 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:17:19.407648 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:17:19.407648 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:17:19.407648 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:17:19.407648 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Nov 02 07:17:19.407648 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:17:19.407648 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:17:19.407648 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:17:19.407648 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:17:19.407648 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:17:19.407648 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:17:19.407648 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:17:19.407648 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:17:19.408532 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Nov 02 07:17:19.408532 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Nov 02 07:17:19.408532 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Nov 02 07:17:19.408532 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Nov 02 07:17:19.408532 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Nov 02 07:17:19.408532 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Nov 02 07:17:19.408532 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 07:17:19.408532 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Nov 02 07:17:19.413133 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 241/486] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Nov 2 07:17:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:17:19.419348 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4806857-d510-4a2c-b735-bd75e091d584 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:19.443642 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6e3829-db1b-40aa-a6b8-ceb06065e887 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:19.446407 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6e3829-db1b-40aa-a6b8-ceb06065e887 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:19.446882 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6e3829-db1b-40aa-a6b8-ceb06065e887 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:19.447309 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6e3829-db1b-40aa-a6b8-ceb06065e887 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:19.448178 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6e3829-db1b-40aa-a6b8-ceb06065e887 None admin] RBAC: Adding query filter params (role.id=4af6f4c514284b8cbddf3e02ec826378, scope.project.id=af04daa468b64160a6176dcab29502a7, user.id=43bb326fb82e4d2c83fb2eaf91860dc8) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:17:19.448608 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6e3829-db1b-40aa-a6b8-ceb06065e887 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:19.450154 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6e3829-db1b-40aa-a6b8-ceb06065e887 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:19.460194 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 246/487] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Nov 2 07:17:19 2022] GET /identity/v3/role_assignments?user.id=43bb326fb82e4d2c83fb2eaf91860dc8&scope.project.id=af04daa468b64160a6176dcab29502a7&role.id=4af6f4c514284b8cbddf3e02ec826378 => generated 608 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:17:19.461219 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6e3829-db1b-40aa-a6b8-ceb06065e887 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:19.572223 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6e3829-db1b-40aa-a6b8-ceb06065e887 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:19.610206 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1f59ea-cdf5-45d1-888c-77646159911e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:19.611195 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1f59ea-cdf5-45d1-888c-77646159911e None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:19.611640 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1f59ea-cdf5-45d1-888c-77646159911e None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:19.612050 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1f59ea-cdf5-45d1-888c-77646159911e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:19.625614 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1f59ea-cdf5-45d1-888c-77646159911e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:17:19.634717 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:19.634900 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:19.636600 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-fb1f59ea-cdf5-45d1-888c-77646159911e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 07:17:19.636600 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:17:19.636600 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:17:19.636600 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:17:19.636600 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:17:19.636600 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:17:19.636600 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:17:19.636600 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:17:19.636600 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:17:19.636600 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:17:19.636600 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:17:19.636600 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:17:19.636600 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:17:19.636600 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:17:19.636600 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:17:19.636600 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:17:19.637879 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:17:19.637879 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:17:19.637879 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:17:19.637879 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:17:19.637879 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:17:19.637879 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:17:19.637879 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:17:19.637879 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:17:19.637879 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:17:19.637879 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:17:19.637879 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:17:19.637879 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:17:19.637879 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:17:19.637879 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:17:19.637879 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:17:19.637879 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:17:19.638984 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:17:19.638984 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:17:19.638984 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:17:19.638984 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:17:19.638984 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:17:19.638984 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:17:19.638984 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:17:19.638984 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:17:19.638984 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:17:19.638984 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:17:19.638984 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 07:17:19.638984 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:17:19.641907 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 247/488] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Nov 2 07:17:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:17:19.654555 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1f59ea-cdf5-45d1-888c-77646159911e None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:19.682933 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4806857-d510-4a2c-b735-bd75e091d584 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:19.720652 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480da205-bbb6-45ba-a198-d6050be54644 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:19.721437 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480da205-bbb6-45ba-a198-d6050be54644 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:19.721810 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480da205-bbb6-45ba-a198-d6050be54644 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:19.722325 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480da205-bbb6-45ba-a198-d6050be54644 None admin] PATH_INFO: `/v3/projects` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:19.723151 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480da205-bbb6-45ba-a198-d6050be54644 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:17:19.723554 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480da205-bbb6-45ba-a198-d6050be54644 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:19.725169 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480da205-bbb6-45ba-a198-d6050be54644 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:19.738825 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 242/489] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Nov 2 07:17:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:17:19.745680 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480da205-bbb6-45ba-a198-d6050be54644 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:19.777078 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1f59ea-cdf5-45d1-888c-77646159911e None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:19.823682 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ebcfec-3274-46b4-ac51-9cfedfa4afec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:19.824528 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ebcfec-3274-46b4-ac51-9cfedfa4afec None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:19.825007 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ebcfec-3274-46b4-ac51-9cfedfa4afec None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:19.825417 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ebcfec-3274-46b4-ac51-9cfedfa4afec None admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:19.826261 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ebcfec-3274-46b4-ac51-9cfedfa4afec None admin] RBAC: Adding query filter params (name=admin) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:17:19.826762 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ebcfec-3274-46b4-ac51-9cfedfa4afec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:19.828311 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ebcfec-3274-46b4-ac51-9cfedfa4afec None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:19.838073 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 248/490] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Nov 2 07:17:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:17:19.839703 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f836dd0e-fca5-4299-90af-56533650f018 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:19.840128 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f836dd0e-fca5-4299-90af-56533650f018 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:19.840536 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f836dd0e-fca5-4299-90af-56533650f018 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:19.842223 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 249/491] 10.222.0.18 () {54 vars in 927 bytes} [Wed Nov 2 07:17:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:17:19.843141 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f836dd0e-fca5-4299-90af-56533650f018 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:19.879485 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480da205-bbb6-45ba-a198-d6050be54644 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:19.920004 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085c1f51-96f5-40e8-9185-a9bf68265d3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:19.920905 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085c1f51-96f5-40e8-9185-a9bf68265d3e None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:19.921433 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085c1f51-96f5-40e8-9185-a9bf68265d3e None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:19.922023 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085c1f51-96f5-40e8-9185-a9bf68265d3e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:19.923012 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085c1f51-96f5-40e8-9185-a9bf68265d3e None admin] RBAC: Adding query filter params (role.id=4af6f4c514284b8cbddf3e02ec826378, scope.project.id=af04daa468b64160a6176dcab29502a7, user.id=b3ec6ecb85f94ef59762877d9ea33ed0) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:17:19.923517 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085c1f51-96f5-40e8-9185-a9bf68265d3e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:19.925187 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085c1f51-96f5-40e8-9185-a9bf68265d3e None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:19.926591 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:19.933379 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 243/492] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Nov 2 07:17:19 2022] GET /identity/v3/role_assignments?user.id=b3ec6ecb85f94ef59762877d9ea33ed0&scope.project.id=af04daa468b64160a6176dcab29502a7&role.id=4af6f4c514284b8cbddf3e02ec826378 => generated 608 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:17:19.934811 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472da93f-913d-4bb7-81b5-1cc2f3895aba None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:19.935194 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472da93f-913d-4bb7-81b5-1cc2f3895aba None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:19.935566 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472da93f-913d-4bb7-81b5-1cc2f3895aba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:19.945082 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:19.957177 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-472da93f-913d-4bb7-81b5-1cc2f3895aba None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:17:19.959235 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-472da93f-913d-4bb7-81b5-1cc2f3895aba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:17:19.989905 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f836dd0e-fca5-4299-90af-56533650f018 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:20.033661 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f52b847-54ea-4e72-8161-c2c059521cfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:20.035143 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f52b847-54ea-4e72-8161-c2c059521cfa None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:20.035598 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f52b847-54ea-4e72-8161-c2c059521cfa None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:20.036015 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f52b847-54ea-4e72-8161-c2c059521cfa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:20.050097 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f52b847-54ea-4e72-8161-c2c059521cfa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:20.053257 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f52b847-54ea-4e72-8161-c2c059521cfa None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:20.071290 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-5f52b847-54ea-4e72-8161-c2c059521cfa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:17:20.071290 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:17:20.071290 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:17:20.071290 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:17:20.071290 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:17:20.071290 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:17:20.071290 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:17:20.071290 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:17:20.071290 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:17:20.071290 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:17:20.071290 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:17:20.071290 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:17:20.071290 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:17:20.071290 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:17:20.071290 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:17:20.071290 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:17:20.072262 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:17:20.072262 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:17:20.072262 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:17:20.072262 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:17:20.072262 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:17:20.072262 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:17:20.072262 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:17:20.072262 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:17:20.072262 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:17:20.072262 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:17:20.072262 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:17:20.072262 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:17:20.072262 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:17:20.072262 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:17:20.072262 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:17:20.072262 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:17:20.072262 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:17:20.073777 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:17:20.073777 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:17:20.073777 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:17:20.073946 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 244/493] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Nov 2 07:17:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:17:20.076035 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 250/494] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Nov 2 07:17:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:17:20.081401 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472da93f-913d-4bb7-81b5-1cc2f3895aba None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:20.237899 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472da93f-913d-4bb7-81b5-1cc2f3895aba None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:20.277592 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f599f539-e493-4754-9853-d3ed7d15d906 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:20.278578 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f599f539-e493-4754-9853-d3ed7d15d906 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:20.279080 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f599f539-e493-4754-9853-d3ed7d15d906 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:20.279584 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f599f539-e493-4754-9853-d3ed7d15d906 None admin] PATH_INFO: `/v3/domains` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:20.280434 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f599f539-e493-4754-9853-d3ed7d15d906 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:17:20.280934 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f599f539-e493-4754-9853-d3ed7d15d906 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:20.282530 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f599f539-e493-4754-9853-d3ed7d15d906 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:20.292814 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:20.295204 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 245/495] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Nov 2 07:17:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:17:20.302208 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f52b847-54ea-4e72-8161-c2c059521cfa None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:20.361694 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f599f539-e493-4754-9853-d3ed7d15d906 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:20.433799 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f52b847-54ea-4e72-8161-c2c059521cfa None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:20.457431 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:20.494477 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd53da51-eab4-4eae-b495-2e971a51d241 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:20.495086 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd53da51-eab4-4eae-b495-2e971a51d241 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:20.495362 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd53da51-eab4-4eae-b495-2e971a51d241 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:20.495628 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd53da51-eab4-4eae-b495-2e971a51d241 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:20.505109 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd53da51-eab4-4eae-b495-2e971a51d241 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:20.506572 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd53da51-eab4-4eae-b495-2e971a51d241 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:20.508651 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f599f539-e493-4754-9853-d3ed7d15d906 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:20.531605 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-bd53da51-eab4-4eae-b495-2e971a51d241 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Nov 02 07:17:20.531605 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:17:20.531605 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:17:20.531605 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:17:20.531605 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:17:20.531605 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:17:20.531605 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:17:20.531605 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:17:20.531605 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:17:20.531605 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:17:20.531605 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:17:20.531605 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:17:20.531605 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Nov 02 07:17:20.531605 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_user(user_id) Nov 02 07:17:20.531605 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Nov 02 07:17:20.531605 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Nov 02 07:17:20.532577 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:17:20.532577 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:17:20.532577 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Nov 02 07:17:20.532577 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:17:20.532577 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Nov 02 07:17:20.532577 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:17:20.532577 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:17:20.532577 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:17:20.532577 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:17:20.532577 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:17:20.532577 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:17:20.532577 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:17:20.532577 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:17:20.532577 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:17:20.532577 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:17:20.532577 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:17:20.532577 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:17:20.532577 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:17:20.533645 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:17:20.533645 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:17:20.533645 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Nov 02 07:17:20.533645 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Nov 02 07:17:20.533645 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Nov 02 07:17:20.533645 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Nov 02 07:17:20.533645 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Nov 02 07:17:20.533645 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Nov 02 07:17:20.533645 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Nov 02 07:17:20.533645 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:17:20.541092 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 251/496] 10.222.0.18 () {56 vars in 1093 bytes} [Wed Nov 2 07:17:20 2022] GET /identity/v3/users/nova => generated 82 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:17:20.542009 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd53da51-eab4-4eae-b495-2e971a51d241 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:20.572553 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a190079a-9a87-4c1e-a114-4ecbb74035f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:20.573243 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a190079a-9a87-4c1e-a114-4ecbb74035f3 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:20.573523 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a190079a-9a87-4c1e-a114-4ecbb74035f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:20.573793 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a190079a-9a87-4c1e-a114-4ecbb74035f3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:20.592556 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a190079a-9a87-4c1e-a114-4ecbb74035f3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:17:20.603806 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-a190079a-9a87-4c1e-a114-4ecbb74035f3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Nov 02 07:17:20.603806 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:17:20.603806 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:17:20.603806 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:17:20.603806 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:17:20.603806 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:17:20.603806 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:17:20.603806 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:17:20.603806 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:17:20.603806 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:17:20.603806 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:17:20.603806 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:17:20.603806 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:17:20.603806 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:17:20.603806 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:17:20.603806 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:17:20.604730 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:17:20.604730 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:17:20.604730 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:17:20.604730 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:17:20.604730 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:17:20.604730 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:17:20.604730 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:17:20.604730 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:17:20.604730 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:17:20.604730 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:17:20.604730 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:17:20.604730 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Nov 02 07:17:20.604730 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:17:20.604730 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:17:20.604730 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:17:20.604730 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:17:20.605533 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:17:20.605533 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:17:20.605533 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:17:20.605533 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:17:20.605533 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:17:20.605533 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:17:20.605533 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:17:20.605533 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:17:20.605533 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:17:20.605533 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:17:20.605533 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Nov 02 07:17:20.605533 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Nov 02 07:17:20.606661 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 246/497] 10.222.0.18 () {56 vars in 1102 bytes} [Wed Nov 2 07:17:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 07:17:20.614496 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a190079a-9a87-4c1e-a114-4ecbb74035f3 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:20.720507 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:20.738545 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd53da51-eab4-4eae-b495-2e971a51d241 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:20.795756 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a190079a-9a87-4c1e-a114-4ecbb74035f3 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:20.804940 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8e9bac-9630-466d-a350-ae62f7206edc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:20.805597 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8e9bac-9630-466d-a350-ae62f7206edc None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:20.805878 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8e9bac-9630-466d-a350-ae62f7206edc None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:20.806144 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8e9bac-9630-466d-a350-ae62f7206edc None admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:20.806943 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8e9bac-9630-466d-a350-ae62f7206edc None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:17:20.807246 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8e9bac-9630-466d-a350-ae62f7206edc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:20.808660 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8e9bac-9630-466d-a350-ae62f7206edc None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:20.844301 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 252/498] 10.222.0.18 () {56 vars in 1133 bytes} [Wed Nov 2 07:17:20 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:17:20.850564 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8e9bac-9630-466d-a350-ae62f7206edc None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:20.857712 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8a7648-190c-4a59-9141-76af010c7690 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:20.858328 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8a7648-190c-4a59-9141-76af010c7690 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:20.859467 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8a7648-190c-4a59-9141-76af010c7690 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:20.859741 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8a7648-190c-4a59-9141-76af010c7690 None admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:20.864902 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8a7648-190c-4a59-9141-76af010c7690 None admin] RBAC: Adding query filter params (name=service) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:17:20.865355 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8a7648-190c-4a59-9141-76af010c7690 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:20.866775 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8a7648-190c-4a59-9141-76af010c7690 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:20.881717 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 247/499] 10.222.0.18 () {56 vars in 1103 bytes} [Wed Nov 2 07:17:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:17:20.887904 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8a7648-190c-4a59-9141-76af010c7690 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:20.907356 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:21.038571 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8e9bac-9630-466d-a350-ae62f7206edc None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:21.087539 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8a7648-190c-4a59-9141-76af010c7690 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:21.109192 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9deae4-58ca-4378-9392-50421e5e7192 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:21.109832 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9deae4-58ca-4378-9392-50421e5e7192 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:21.110169 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:21.110973 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9deae4-58ca-4378-9392-50421e5e7192 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:21.111242 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9deae4-58ca-4378-9392-50421e5e7192 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:21.134067 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9deae4-58ca-4378-9392-50421e5e7192 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:21.136612 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9deae4-58ca-4378-9392-50421e5e7192 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:21.152503 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f60bdb1-4522-425f-b015-47d3d331e9b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:21.153453 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f60bdb1-4522-425f-b015-47d3d331e9b5 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:21.153929 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f60bdb1-4522-425f-b015-47d3d331e9b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:21.154422 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f60bdb1-4522-425f-b015-47d3d331e9b5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:21.160097 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-ea9deae4-58ca-4378-9392-50421e5e7192 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:17:21.160097 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:17:21.160097 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:17:21.160097 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:17:21.160097 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:17:21.160097 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:17:21.160097 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:17:21.160097 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:17:21.160097 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:17:21.160097 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:17:21.160097 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:17:21.160097 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:17:21.160097 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:17:21.160097 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:17:21.160097 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:17:21.160097 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:17:21.161606 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:17:21.161606 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:17:21.161606 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:17:21.161606 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:17:21.161606 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:17:21.161606 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:17:21.161606 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:17:21.161606 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:17:21.161606 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:17:21.161606 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:17:21.161606 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:17:21.161606 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:17:21.161606 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:17:21.161606 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:17:21.161606 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:17:21.161606 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:17:21.161606 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:17:21.162649 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:17:21.162649 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:17:21.162649 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:17:21.170899 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 253/500] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Nov 2 07:17:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 321 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:17:21.174824 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f60bdb1-4522-425f-b015-47d3d331e9b5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:21.176274 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9deae4-58ca-4378-9392-50421e5e7192 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:21.179058 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f60bdb1-4522-425f-b015-47d3d331e9b5 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:21.207904 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-2f60bdb1-4522-425f-b015-47d3d331e9b5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:17:21.207904 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:17:21.207904 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:17:21.207904 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:17:21.207904 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:17:21.207904 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:17:21.207904 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:17:21.207904 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:17:21.207904 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:17:21.207904 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:17:21.207904 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:17:21.207904 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:17:21.207904 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:17:21.207904 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:17:21.207904 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:17:21.207904 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:17:21.209044 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:17:21.209044 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:17:21.209044 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:17:21.209044 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:17:21.209044 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:17:21.209044 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:17:21.209044 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:17:21.209044 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Nov 02 07:17:21.209044 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:17:21.209044 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:17:21.209044 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:17:21.209044 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:17:21.209044 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:17:21.209044 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:17:21.209044 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:17:21.209044 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:17:21.209044 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:17:21.209907 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:17:21.209907 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:17:21.209907 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Nov 02 07:17:21.216395 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 248/501] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Nov 2 07:17:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 324 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:17:21.218082 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f60bdb1-4522-425f-b015-47d3d331e9b5 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:21.293770 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:21.319778 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9deae4-58ca-4378-9392-50421e5e7192 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:21.368626 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c40e288-21f1-4b87-a7b6-1fdc649df461 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:21.369152 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c40e288-21f1-4b87-a7b6-1fdc649df461 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:21.369430 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c40e288-21f1-4b87-a7b6-1fdc649df461 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:21.369682 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c40e288-21f1-4b87-a7b6-1fdc649df461 None admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:21.370312 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c40e288-21f1-4b87-a7b6-1fdc649df461 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:17:21.370897 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c40e288-21f1-4b87-a7b6-1fdc649df461 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:21.372196 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c40e288-21f1-4b87-a7b6-1fdc649df461 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:21.392836 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 254/502] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Nov 2 07:17:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:17:21.394495 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccd4a6e-211d-439d-8835-b923de30f30f None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:21.394925 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccd4a6e-211d-439d-8835-b923de30f30f None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:21.395105 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f60bdb1-4522-425f-b015-47d3d331e9b5 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:21.396152 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccd4a6e-211d-439d-8835-b923de30f30f None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:21.402117 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 255/503] 10.222.0.18 () {54 vars in 927 bytes} [Wed Nov 2 07:17:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:17:21.403173 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccd4a6e-211d-439d-8835-b923de30f30f None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:21.439740 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402bbe1d-1e1d-4cc4-9145-f823a9086a1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:21.441352 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402bbe1d-1e1d-4cc4-9145-f823a9086a1b None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:21.441732 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402bbe1d-1e1d-4cc4-9145-f823a9086a1b None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:21.442112 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402bbe1d-1e1d-4cc4-9145-f823a9086a1b None admin] PATH_INFO: `/v3/domains` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:21.443546 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402bbe1d-1e1d-4cc4-9145-f823a9086a1b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:17:21.443949 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402bbe1d-1e1d-4cc4-9145-f823a9086a1b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:21.445429 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402bbe1d-1e1d-4cc4-9145-f823a9086a1b None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:21.465011 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 249/504] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Nov 2 07:17:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:17:21.465011 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49ed1c2-1a85-4ebb-959d-305b1f8059e4 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:21.465011 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49ed1c2-1a85-4ebb-959d-305b1f8059e4 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:21.465011 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49ed1c2-1a85-4ebb-959d-305b1f8059e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:21.485690 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-c49ed1c2-1a85-4ebb-959d-305b1f8059e4 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:17:21.487354 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-c49ed1c2-1a85-4ebb-959d-305b1f8059e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:17:21.521686 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccd4a6e-211d-439d-8835-b923de30f30f None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:21.525216 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:21.560784 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b899bc-38be-4330-9133-e35edc1fba3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:21.561413 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b899bc-38be-4330-9133-e35edc1fba3f None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:21.561691 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b899bc-38be-4330-9133-e35edc1fba3f None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:21.562011 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b899bc-38be-4330-9133-e35edc1fba3f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:21.569821 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b899bc-38be-4330-9133-e35edc1fba3f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:21.571617 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b899bc-38be-4330-9133-e35edc1fba3f None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:21.590394 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 250/505] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Nov 2 07:17:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:17:21.590394 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49ed1c2-1a85-4ebb-959d-305b1f8059e4 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:21.592988 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-77b899bc-38be-4330-9133-e35edc1fba3f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 07:17:21.592988 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:17:21.592988 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:17:21.592988 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:17:21.592988 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:17:21.592988 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:17:21.592988 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:17:21.592988 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:17:21.592988 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:17:21.592988 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:17:21.592988 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:17:21.592988 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:17:21.592988 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Nov 02 07:17:21.592988 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_project(project_id) Nov 02 07:17:21.592988 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Nov 02 07:17:21.593942 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Nov 02 07:17:21.593942 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:17:21.593942 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:17:21.593942 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:17:21.593942 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:17:21.593942 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:17:21.593942 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:17:21.593942 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:17:21.593942 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:17:21.593942 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:17:21.593942 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:17:21.593942 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:17:21.593942 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:17:21.593942 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:17:21.593942 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:17:21.593942 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:17:21.593942 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:17:21.594858 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Nov 02 07:17:21.594858 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Nov 02 07:17:21.594858 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Nov 02 07:17:21.594858 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Nov 02 07:17:21.594858 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Nov 02 07:17:21.594858 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Nov 02 07:17:21.594858 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 07:17:21.594858 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:17:21.604967 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 256/506] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Nov 2 07:17:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:17:21.605734 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b899bc-38be-4330-9133-e35edc1fba3f None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:21.612911 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:21.768464 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49ed1c2-1a85-4ebb-959d-305b1f8059e4 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:21.800218 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b899bc-38be-4330-9133-e35edc1fba3f None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:21.808524 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0179dcdd-e591-4510-bcb0-1cf6fdc81f71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:21.809156 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0179dcdd-e591-4510-bcb0-1cf6fdc81f71 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:21.809437 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0179dcdd-e591-4510-bcb0-1cf6fdc81f71 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:21.809708 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0179dcdd-e591-4510-bcb0-1cf6fdc81f71 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:21.817793 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0179dcdd-e591-4510-bcb0-1cf6fdc81f71 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:21.819209 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0179dcdd-e591-4510-bcb0-1cf6fdc81f71 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:21.834300 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-0179dcdd-e591-4510-bcb0-1cf6fdc81f71 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Nov 02 07:17:21.834300 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:17:21.834300 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:17:21.834300 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:17:21.834300 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:17:21.834300 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:17:21.834300 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:17:21.834300 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:17:21.834300 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:17:21.834300 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:17:21.834300 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:17:21.834300 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:17:21.834300 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Nov 02 07:17:21.834300 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_user(user_id) Nov 02 07:17:21.834300 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Nov 02 07:17:21.834300 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Nov 02 07:17:21.835237 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:17:21.835237 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:17:21.835237 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Nov 02 07:17:21.835237 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:17:21.835237 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Nov 02 07:17:21.835237 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:17:21.835237 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:17:21.835237 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:17:21.835237 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:17:21.835237 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:17:21.835237 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:17:21.835237 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Nov 02 07:17:21.835237 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:17:21.835237 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:17:21.835237 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:17:21.835237 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:17:21.835237 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:17:21.835237 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:17:21.836154 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:17:21.836154 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:17:21.836154 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Nov 02 07:17:21.836154 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Nov 02 07:17:21.836154 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Nov 02 07:17:21.836154 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Nov 02 07:17:21.836154 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Nov 02 07:17:21.836154 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Nov 02 07:17:21.836154 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Nov 02 07:17:21.836154 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Nov 02 07:17:21.836671 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 251/507] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Nov 2 07:17:21 2022] GET /identity/v3/users/swift => generated 83 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:17:21.841677 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0179dcdd-e591-4510-bcb0-1cf6fdc81f71 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:21.876421 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:21.880997 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90757b98-49a5-44a8-8c18-70ae533b541a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:21.881629 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90757b98-49a5-44a8-8c18-70ae533b541a None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:21.881910 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90757b98-49a5-44a8-8c18-70ae533b541a None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:21.882181 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90757b98-49a5-44a8-8c18-70ae533b541a None admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:21.882915 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90757b98-49a5-44a8-8c18-70ae533b541a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:17:21.883215 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90757b98-49a5-44a8-8c18-70ae533b541a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:21.884654 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90757b98-49a5-44a8-8c18-70ae533b541a None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:21.912965 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 257/508] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Nov 2 07:17:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:17:21.920469 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90757b98-49a5-44a8-8c18-70ae533b541a None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:21.998858 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0179dcdd-e591-4510-bcb0-1cf6fdc81f71 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:22.004514 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:22.037234 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b74758-560b-4347-b6f6-e8acb13da4b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:22.038429 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b74758-560b-4347-b6f6-e8acb13da4b1 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:22.038953 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b74758-560b-4347-b6f6-e8acb13da4b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:22.039433 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b74758-560b-4347-b6f6-e8acb13da4b1 None admin] PATH_INFO: `/v3/users` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:22.040353 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b74758-560b-4347-b6f6-e8acb13da4b1 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:17:22.040904 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b74758-560b-4347-b6f6-e8acb13da4b1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:22.042675 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b74758-560b-4347-b6f6-e8acb13da4b1 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:22.048650 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90757b98-49a5-44a8-8c18-70ae533b541a None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:22.062806 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 252/509] 10.222.0.18 () {56 vars in 1135 bytes} [Wed Nov 2 07:17:21 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:17:22.064037 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b74758-560b-4347-b6f6-e8acb13da4b1 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:22.086770 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e138978-2c9c-4b96-9e5d-6fc208621c16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:22.087435 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e138978-2c9c-4b96-9e5d-6fc208621c16 None admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:22.087714 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e138978-2c9c-4b96-9e5d-6fc208621c16 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:22.087989 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e138978-2c9c-4b96-9e5d-6fc208621c16 None admin] PATH_INFO: `/v3/projects/af04daa468b64160a6176dcab29502a7/users/69e48cba3f804cf08b2477f01ae95179/roles/c4bf019fc6c44c71a6c984f6d0bab75f` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:22.089903 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e138978-2c9c-4b96-9e5d-6fc208621c16 None admin] RBAC: Authorizing `identity:create_grant(project_id=af04daa468b64160a6176dcab29502a7, user_id=69e48cba3f804cf08b2477f01ae95179, role_id=c4bf019fc6c44c71a6c984f6d0bab75f)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:22.091362 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e138978-2c9c-4b96-9e5d-6fc208621c16 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:22.110765 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 258/510] 10.222.0.18 () {58 vars in 1439 bytes} [Wed Nov 2 07:17:21 2022] PUT /identity/v3/projects/af04daa468b64160a6176dcab29502a7/users/69e48cba3f804cf08b2477f01ae95179/roles/c4bf019fc6c44c71a6c984f6d0bab75f => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:17:22.110765 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e138978-2c9c-4b96-9e5d-6fc208621c16 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:22.192553 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:22.229435 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b74758-560b-4347-b6f6-e8acb13da4b1 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:22.239105 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e138978-2c9c-4b96-9e5d-6fc208621c16 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:22.268688 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:22.280232 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c7d79b-e6d8-4c6b-9351-bd811fbc55a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:22.281087 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c7d79b-e6d8-4c6b-9351-bd811fbc55a6 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:22.281433 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c7d79b-e6d8-4c6b-9351-bd811fbc55a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:22.281750 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c7d79b-e6d8-4c6b-9351-bd811fbc55a6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:22.288419 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c74a0f-b146-4a83-bf39-2693f5faf350 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:22.289324 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c74a0f-b146-4a83-bf39-2693f5faf350 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:22.289606 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c74a0f-b146-4a83-bf39-2693f5faf350 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:22.289876 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c74a0f-b146-4a83-bf39-2693f5faf350 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:22.296665 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c7d79b-e6d8-4c6b-9351-bd811fbc55a6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:22.298143 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c7d79b-e6d8-4c6b-9351-bd811fbc55a6 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:22.316545 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-d2c7d79b-e6d8-4c6b-9351-bd811fbc55a6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:17:22.316545 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:17:22.316545 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:17:22.316545 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:17:22.316545 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:17:22.316545 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:17:22.316545 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:17:22.316545 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:17:22.316545 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:17:22.316545 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:17:22.316545 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:17:22.316545 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:17:22.316545 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:17:22.316545 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:17:22.316545 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:17:22.316545 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:17:22.318782 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:17:22.318782 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:17:22.318782 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:17:22.318782 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:17:22.318782 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:17:22.318782 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:17:22.318782 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:17:22.318782 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Nov 02 07:17:22.318782 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:17:22.318782 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:17:22.318782 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:17:22.318782 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:17:22.318782 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:17:22.318782 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:17:22.318782 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:17:22.318782 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:17:22.318782 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:17:22.319686 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:17:22.319686 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:17:22.319686 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Nov 02 07:17:22.319686 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 253/511] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Nov 2 07:17:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:17:22.319686 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ce7070-98bf-4f97-881f-504c1953edd9 None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:22.319968 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ce7070-98bf-4f97-881f-504c1953edd9 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:22.320037 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ce7070-98bf-4f97-881f-504c1953edd9 None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:22.321731 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 254/512] 10.222.0.18 () {54 vars in 927 bytes} [Wed Nov 2 07:17:22 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:17:22.326727 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e168a0-5c00-4bfa-8aa6-6031f1baca1b None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:22.327164 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e168a0-5c00-4bfa-8aa6-6031f1baca1b None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:22.327438 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e168a0-5c00-4bfa-8aa6-6031f1baca1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:22.347458 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-68e168a0-5c00-4bfa-8aa6-6031f1baca1b None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:17:22.348690 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-68e168a0-5c00-4bfa-8aa6-6031f1baca1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:17:22.376391 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c74a0f-b146-4a83-bf39-2693f5faf350 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:22.378782 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c74a0f-b146-4a83-bf39-2693f5faf350 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:22.430592 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 255/513] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Nov 2 07:17:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Nov 02 07:17:22.431452 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e168a0-5c00-4bfa-8aa6-6031f1baca1b None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:22.514644 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:22.556148 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 259/514] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Nov 2 07:17:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Nov 02 07:17:22.563376 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c74a0f-b146-4a83-bf39-2693f5faf350 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:22.595386 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:22.609977 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e168a0-5c00-4bfa-8aa6-6031f1baca1b None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:22.658387 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0597ee-1bad-4faa-9071-133a84c4b919 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:22.658387 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0597ee-1bad-4faa-9071-133a84c4b919 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:22.658387 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0597ee-1bad-4faa-9071-133a84c4b919 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:22.658387 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0597ee-1bad-4faa-9071-133a84c4b919 None admin] PATH_INFO: `/v3/domains` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:22.658787 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0597ee-1bad-4faa-9071-133a84c4b919 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:17:22.658787 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0597ee-1bad-4faa-9071-133a84c4b919 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:22.658787 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0597ee-1bad-4faa-9071-133a84c4b919 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:22.673385 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 256/515] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Nov 2 07:17:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:17:22.675157 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13d7973-aab8-4bdc-97fc-4b51cd681eef None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:22.675446 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13d7973-aab8-4bdc-97fc-4b51cd681eef None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:22.675717 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13d7973-aab8-4bdc-97fc-4b51cd681eef None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:22.678197 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 257/516] 10.222.0.18 () {54 vars in 927 bytes} [Wed Nov 2 07:17:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:17:22.680486 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13d7973-aab8-4bdc-97fc-4b51cd681eef None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:22.697518 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c74a0f-b146-4a83-bf39-2693f5faf350 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:22.738029 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878d8423-f794-4d7c-9382-4531da8a3ead None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:22.738694 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878d8423-f794-4d7c-9382-4531da8a3ead None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:22.738973 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878d8423-f794-4d7c-9382-4531da8a3ead None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:22.740007 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878d8423-f794-4d7c-9382-4531da8a3ead None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:22.751071 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878d8423-f794-4d7c-9382-4531da8a3ead None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:22.752513 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878d8423-f794-4d7c-9382-4531da8a3ead None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:22.769756 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-878d8423-f794-4d7c-9382-4531da8a3ead None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:17:22.769756 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:17:22.769756 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:17:22.769756 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:17:22.769756 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:17:22.769756 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:17:22.769756 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:17:22.769756 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:17:22.769756 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:17:22.769756 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:17:22.769756 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:17:22.769756 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:17:22.769756 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:17:22.769756 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:17:22.769756 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:17:22.769756 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:17:22.771144 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:17:22.771144 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:17:22.771144 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:17:22.771144 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:17:22.771144 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:17:22.771144 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:17:22.771144 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:17:22.771144 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:17:22.771144 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:17:22.771144 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:17:22.771144 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:17:22.771144 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:17:22.771144 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:17:22.771144 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:17:22.771144 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:17:22.771144 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:17:22.771144 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:17:22.772036 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:17:22.772036 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:17:22.772036 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:17:22.772036 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 260/517] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Nov 2 07:17:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:17:22.773817 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f87f1f-6002-4ae5-a636-cf26026c4a8d None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:22.773817 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f87f1f-6002-4ae5-a636-cf26026c4a8d None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:22.773817 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f87f1f-6002-4ae5-a636-cf26026c4a8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:22.794203 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-38f87f1f-6002-4ae5-a636-cf26026c4a8d None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:17:22.794487 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13d7973-aab8-4bdc-97fc-4b51cd681eef None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:22.795497 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-38f87f1f-6002-4ae5-a636-cf26026c4a8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:17:22.813530 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:22.833137 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e52023f-26d9-4e88-a361-8be33e91695e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:22.833830 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e52023f-26d9-4e88-a361-8be33e91695e None admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:22.834196 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e52023f-26d9-4e88-a361-8be33e91695e None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:22.834547 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e52023f-26d9-4e88-a361-8be33e91695e None admin] PATH_INFO: `/v3/users` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:22.835795 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e52023f-26d9-4e88-a361-8be33e91695e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:22.836599 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e52023f-26d9-4e88-a361-8be33e91695e None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:22.870385 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 258/518] 10.222.0.18 () {60 vars in 1132 bytes} [Wed Nov 2 07:17:22 2022] POST /identity/v3/users => generated 254 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:17:22.870385 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e52023f-26d9-4e88-a361-8be33e91695e None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:22.870994 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 261/519] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Nov 2 07:17:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:17:22.870994 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f87f1f-6002-4ae5-a636-cf26026c4a8d None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:22.879959 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:23.074004 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e52023f-26d9-4e88-a361-8be33e91695e None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:23.080329 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f87f1f-6002-4ae5-a636-cf26026c4a8d None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:23.112915 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df084289-10a3-4984-a435-8fb02dae2ed8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:23.113558 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df084289-10a3-4984-a435-8fb02dae2ed8 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:23.113839 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df084289-10a3-4984-a435-8fb02dae2ed8 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:23.114145 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df084289-10a3-4984-a435-8fb02dae2ed8 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:23.114947 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df084289-10a3-4984-a435-8fb02dae2ed8 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:23.116324 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df084289-10a3-4984-a435-8fb02dae2ed8 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:23.122849 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da053ae1-d506-44ea-b3e7-f98077193add None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:23.123746 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da053ae1-d506-44ea-b3e7-f98077193add None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:23.124221 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da053ae1-d506-44ea-b3e7-f98077193add None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:23.124706 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da053ae1-d506-44ea-b3e7-f98077193add None admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:23.125550 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da053ae1-d506-44ea-b3e7-f98077193add None admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:17:23.126045 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da053ae1-d506-44ea-b3e7-f98077193add None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:23.127627 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da053ae1-d506-44ea-b3e7-f98077193add None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:23.128867 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-df084289-10a3-4984-a435-8fb02dae2ed8 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Nov 02 07:17:23.128867 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:17:23.128867 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Nov 02 07:17:23.128867 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Nov 02 07:17:23.128867 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Nov 02 07:17:23.128867 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Nov 02 07:17:23.128867 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Nov 02 07:17:23.128867 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:17:23.128867 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Nov 02 07:17:23.128867 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Nov 02 07:17:23.128867 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 07:17:23.128867 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Nov 02 07:17:23.128867 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:17:23.128867 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:17:23.128867 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:17:23.128867 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:17:23.128867 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:17:23.130521 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:17:23.130521 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:17:23.130521 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:17:23.130521 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:17:23.130521 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:17:23.130521 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:17:23.130521 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Nov 02 07:17:23.130521 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_service(service_id) Nov 02 07:17:23.130521 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Nov 02 07:17:23.130521 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Nov 02 07:17:23.130521 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:17:23.130521 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:17:23.130521 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:17:23.130521 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:17:23.130521 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:17:23.130521 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:17:23.132142 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:17:23.132142 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Nov 02 07:17:23.132142 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:17:23.132142 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:17:23.132142 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:17:23.132142 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:17:23.132142 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:17:23.132142 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:17:23.132142 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:17:23.132142 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:17:23.132142 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Nov 02 07:17:23.132142 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:17:23.132142 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Nov 02 07:17:23.132142 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Nov 02 07:17:23.132142 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 259/520] 10.222.0.18 () {56 vars in 1129 bytes} [Wed Nov 2 07:17:22 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:17:23.137038 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df084289-10a3-4984-a435-8fb02dae2ed8 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:23.149838 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 262/521] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Nov 2 07:17:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:17:23.151531 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da053ae1-d506-44ea-b3e7-f98077193add None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:23.182031 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:23.230442 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:23.252674 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df084289-10a3-4984-a435-8fb02dae2ed8 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:23.279834 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da053ae1-d506-44ea-b3e7-f98077193add None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:23.292887 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03732527-0bd7-4443-b1bd-721f3fae8828 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:23.293576 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03732527-0bd7-4443-b1bd-721f3fae8828 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:23.293922 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03732527-0bd7-4443-b1bd-721f3fae8828 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:23.294277 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03732527-0bd7-4443-b1bd-721f3fae8828 None admin] PATH_INFO: `/v3/services` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:23.295063 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03732527-0bd7-4443-b1bd-721f3fae8828 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:17:23.295437 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03732527-0bd7-4443-b1bd-721f3fae8828 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:23.297031 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03732527-0bd7-4443-b1bd-721f3fae8828 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:23.302809 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 260/522] 10.222.0.18 () {56 vars in 1124 bytes} [Wed Nov 2 07:17:23 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:17:23.303810 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03732527-0bd7-4443-b1bd-721f3fae8828 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:23.318545 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31d8089-5579-49dc-b8df-e31800f2ceab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:23.319552 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31d8089-5579-49dc-b8df-e31800f2ceab None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:23.320151 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31d8089-5579-49dc-b8df-e31800f2ceab None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:23.320423 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31d8089-5579-49dc-b8df-e31800f2ceab None admin] PATH_INFO: `/v3/projects/service` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:23.327846 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31d8089-5579-49dc-b8df-e31800f2ceab None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:23.329326 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31d8089-5579-49dc-b8df-e31800f2ceab None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:23.343237 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-f31d8089-5579-49dc-b8df-e31800f2ceab None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 07:17:23.343237 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:17:23.343237 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:17:23.343237 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:17:23.343237 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:17:23.343237 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:17:23.343237 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:17:23.343237 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:17:23.343237 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:17:23.343237 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:17:23.343237 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:17:23.343237 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:17:23.343237 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Nov 02 07:17:23.343237 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_project(project_id) Nov 02 07:17:23.343237 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Nov 02 07:17:23.344153 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Nov 02 07:17:23.344153 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:17:23.344153 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:17:23.344153 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:17:23.344153 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:17:23.344153 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:17:23.344153 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:17:23.344153 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:17:23.344153 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:17:23.344153 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:17:23.344153 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:17:23.344153 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:17:23.344153 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:17:23.344153 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:17:23.344153 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:17:23.344153 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:17:23.344153 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:17:23.345020 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Nov 02 07:17:23.345020 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Nov 02 07:17:23.345020 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Nov 02 07:17:23.345020 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Nov 02 07:17:23.345020 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Nov 02 07:17:23.345020 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Nov 02 07:17:23.345020 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 07:17:23.345020 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:17:23.349185 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 263/523] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Nov 2 07:17:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:17:23.350053 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31d8089-5579-49dc-b8df-e31800f2ceab None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:23.427829 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03732527-0bd7-4443-b1bd-721f3fae8828 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:23.429600 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:23.470421 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb899fb8-1efe-4429-ad36-efc7af624ac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:23.471067 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb899fb8-1efe-4429-ad36-efc7af624ac7 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:23.471350 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb899fb8-1efe-4429-ad36-efc7af624ac7 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:23.471623 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb899fb8-1efe-4429-ad36-efc7af624ac7 None admin] PATH_INFO: `/v3/services/network` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:23.472304 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb899fb8-1efe-4429-ad36-efc7af624ac7 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:23.474565 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb899fb8-1efe-4429-ad36-efc7af624ac7 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:23.483662 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31d8089-5579-49dc-b8df-e31800f2ceab None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:23.484472 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-fb899fb8-1efe-4429-ad36-efc7af624ac7 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Nov 02 07:17:23.484472 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:17:23.484472 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Nov 02 07:17:23.484472 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Nov 02 07:17:23.484472 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Nov 02 07:17:23.484472 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Nov 02 07:17:23.484472 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Nov 02 07:17:23.484472 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:17:23.484472 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Nov 02 07:17:23.484472 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Nov 02 07:17:23.484472 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 07:17:23.484472 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Nov 02 07:17:23.484472 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:17:23.484472 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:17:23.484472 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:17:23.484472 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:17:23.484472 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:17:23.485454 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:17:23.485454 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:17:23.485454 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:17:23.485454 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:17:23.485454 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:17:23.485454 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:17:23.485454 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Nov 02 07:17:23.485454 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_service(service_id) Nov 02 07:17:23.485454 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Nov 02 07:17:23.485454 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Nov 02 07:17:23.485454 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:17:23.485454 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:17:23.485454 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:17:23.485454 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:17:23.485454 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:17:23.485454 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:17:23.486283 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:17:23.486283 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Nov 02 07:17:23.486283 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:17:23.486283 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:17:23.486283 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:17:23.486283 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:17:23.486283 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:17:23.486283 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:17:23.486283 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:17:23.486283 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:17:23.486283 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Nov 02 07:17:23.486283 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:17:23.486283 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Nov 02 07:17:23.486283 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Nov 02 07:17:23.491810 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 261/524] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Nov 2 07:17:23 2022] GET /identity/v3/services/network => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:17:23.492884 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb899fb8-1efe-4429-ad36-efc7af624ac7 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:23.518548 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:23.533245 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b640129-208a-4105-9a72-c6dba4716529 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:23.534341 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b640129-208a-4105-9a72-c6dba4716529 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:23.535669 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b640129-208a-4105-9a72-c6dba4716529 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:23.536203 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b640129-208a-4105-9a72-c6dba4716529 None admin] PATH_INFO: `/v3/services` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:23.537109 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b640129-208a-4105-9a72-c6dba4716529 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:17:23.537611 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b640129-208a-4105-9a72-c6dba4716529 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:23.539166 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b640129-208a-4105-9a72-c6dba4716529 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:23.547219 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 264/525] 10.222.0.18 () {56 vars in 1124 bytes} [Wed Nov 2 07:17:23 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:17:23.548983 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b640129-208a-4105-9a72-c6dba4716529 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:23.618545 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb899fb8-1efe-4429-ad36-efc7af624ac7 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:23.673069 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3aa6dc6-de96-4c88-aac5-090e3b91c935 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:23.673927 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3aa6dc6-de96-4c88-aac5-090e3b91c935 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:23.674470 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3aa6dc6-de96-4c88-aac5-090e3b91c935 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:23.675204 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3aa6dc6-de96-4c88-aac5-090e3b91c935 None admin] PATH_INFO: `/v3/projects` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:23.675736 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3aa6dc6-de96-4c88-aac5-090e3b91c935 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:17:23.676143 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3aa6dc6-de96-4c88-aac5-090e3b91c935 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:23.678556 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3aa6dc6-de96-4c88-aac5-090e3b91c935 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:23.690276 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b640129-208a-4105-9a72-c6dba4716529 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:23.694523 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 262/526] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Nov 2 07:17:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:17:23.705360 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3aa6dc6-de96-4c88-aac5-090e3b91c935 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:23.741624 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dacc646-928f-4116-a2be-bf7ae6e8e5ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:23.742255 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dacc646-928f-4116-a2be-bf7ae6e8e5ed None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:23.742571 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dacc646-928f-4116-a2be-bf7ae6e8e5ed None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:23.742844 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dacc646-928f-4116-a2be-bf7ae6e8e5ed None admin] PATH_INFO: `/v3/services` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:23.745380 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dacc646-928f-4116-a2be-bf7ae6e8e5ed None admin] RBAC: Adding query filter params (name=network) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:17:23.745686 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dacc646-928f-4116-a2be-bf7ae6e8e5ed None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:23.748270 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dacc646-928f-4116-a2be-bf7ae6e8e5ed None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:23.759367 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 265/527] 10.222.0.18 () {56 vars in 1112 bytes} [Wed Nov 2 07:17:23 2022] GET /identity/v3/services?name=network => generated 124 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:17:23.764379 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:23.765708 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dacc646-928f-4116-a2be-bf7ae6e8e5ed None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:23.859272 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:23.882659 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3aa6dc6-de96-4c88-aac5-090e3b91c935 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:23.920750 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dacc646-928f-4116-a2be-bf7ae6e8e5ed None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:23.923431 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aaabd04-a6dc-4766-8dcd-17b9666c6545 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:23.924326 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaabd04-a6dc-4766-8dcd-17b9666c6545 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:23.924808 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaabd04-a6dc-4766-8dcd-17b9666c6545 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:23.925328 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaabd04-a6dc-4766-8dcd-17b9666c6545 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:23.926347 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aaabd04-a6dc-4766-8dcd-17b9666c6545 None admin] RBAC: Adding query filter params (role.id=4af6f4c514284b8cbddf3e02ec826378, scope.project.id=af04daa468b64160a6176dcab29502a7, user.id=f9116f47889f4db39e0203d6cdae1593) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:17:23.927630 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aaabd04-a6dc-4766-8dcd-17b9666c6545 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:23.929440 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aaabd04-a6dc-4766-8dcd-17b9666c6545 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:23.936660 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 263/528] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Nov 2 07:17:23 2022] GET /identity/v3/role_assignments?user.id=f9116f47889f4db39e0203d6cdae1593&scope.project.id=af04daa468b64160a6176dcab29502a7&role.id=4af6f4c514284b8cbddf3e02ec826378 => generated 608 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:17:23.966771 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d88aba5-87d9-4c6a-8148-5579f3dc695f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:23.967405 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d88aba5-87d9-4c6a-8148-5579f3dc695f None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:23.967677 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d88aba5-87d9-4c6a-8148-5579f3dc695f None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:23.967944 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d88aba5-87d9-4c6a-8148-5579f3dc695f None admin] PATH_INFO: `/v3/services` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:23.968588 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d88aba5-87d9-4c6a-8148-5579f3dc695f None admin] RBAC: Adding query filter params (type=network) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:17:23.968930 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d88aba5-87d9-4c6a-8148-5579f3dc695f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:23.970285 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d88aba5-87d9-4c6a-8148-5579f3dc695f None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:23.982778 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 266/529] 10.222.0.18 () {56 vars in 1112 bytes} [Wed Nov 2 07:17:23 2022] GET /identity/v3/services?type=network => generated 124 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:17:24.513288 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb2df5d-4f96-4215-aa29-96f28f05e201 None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:24.514330 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb2df5d-4f96-4215-aa29-96f28f05e201 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:24.514871 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb2df5d-4f96-4215-aa29-96f28f05e201 None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:24.516670 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 264/530] 10.222.0.18 () {54 vars in 927 bytes} [Wed Nov 2 07:17:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:17:24.524108 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7410a37-12b2-49f2-9569-3edc5d393e6f None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:24.524958 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7410a37-12b2-49f2-9569-3edc5d393e6f None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:24.525435 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7410a37-12b2-49f2-9569-3edc5d393e6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:24.545788 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-d7410a37-12b2-49f2-9569-3edc5d393e6f None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:17:24.547336 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-d7410a37-12b2-49f2-9569-3edc5d393e6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:17:24.623209 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 267/531] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Nov 2 07:17:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:17:24.944788 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb2df5d-4f96-4215-aa29-96f28f05e201 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:25.046521 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:25.058652 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb2df5d-4f96-4215-aa29-96f28f05e201 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:25.096445 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5544afd6-1b17-471d-a8a2-d0f50c0e6004 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:25.097512 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5544afd6-1b17-471d-a8a2-d0f50c0e6004 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:25.098069 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5544afd6-1b17-471d-a8a2-d0f50c0e6004 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:25.098779 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5544afd6-1b17-471d-a8a2-d0f50c0e6004 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:25.111345 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5544afd6-1b17-471d-a8a2-d0f50c0e6004 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:17:25.117123 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-5544afd6-1b17-471d-a8a2-d0f50c0e6004 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 07:17:25.117123 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:17:25.117123 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:17:25.117123 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:17:25.117123 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:17:25.117123 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:17:25.117123 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:17:25.117123 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:17:25.117123 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:17:25.117123 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:17:25.117123 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:17:25.117123 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:17:25.117123 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:17:25.117123 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:17:25.117123 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:17:25.117123 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:17:25.118041 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:17:25.118041 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:17:25.118041 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:17:25.118041 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:17:25.118041 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:17:25.118041 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:17:25.118041 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:17:25.118041 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:17:25.118041 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:17:25.118041 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:17:25.118041 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:17:25.118041 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Nov 02 07:17:25.118041 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:17:25.118041 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:17:25.118041 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:17:25.118041 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:17:25.118947 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:17:25.118947 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:17:25.118947 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:17:25.118947 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:17:25.118947 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:17:25.118947 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:17:25.118947 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:17:25.118947 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:17:25.118947 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:17:25.118947 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:17:25.118947 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 07:17:25.118947 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Nov 02 07:17:25.121763 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 265/532] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Nov 2 07:17:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:17:25.127959 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7410a37-12b2-49f2-9569-3edc5d393e6f None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:25.182685 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:25.244024 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7410a37-12b2-49f2-9569-3edc5d393e6f None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:25.281393 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05b1f67-dd2a-4ea6-b189-7a14b747083d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:25.282384 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05b1f67-dd2a-4ea6-b189-7a14b747083d None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:25.282891 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05b1f67-dd2a-4ea6-b189-7a14b747083d None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:25.283371 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05b1f67-dd2a-4ea6-b189-7a14b747083d None admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:25.284225 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05b1f67-dd2a-4ea6-b189-7a14b747083d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:17:25.284747 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05b1f67-dd2a-4ea6-b189-7a14b747083d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:25.287072 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05b1f67-dd2a-4ea6-b189-7a14b747083d None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:25.296201 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 268/533] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Nov 2 07:17:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:17:25.302977 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5544afd6-1b17-471d-a8a2-d0f50c0e6004 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:25.417292 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5544afd6-1b17-471d-a8a2-d0f50c0e6004 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:25.452875 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59690f5-edcb-4339-91ac-dc430e33666c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:25.453842 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59690f5-edcb-4339-91ac-dc430e33666c None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:25.454330 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59690f5-edcb-4339-91ac-dc430e33666c None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:25.454829 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59690f5-edcb-4339-91ac-dc430e33666c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:25.467928 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59690f5-edcb-4339-91ac-dc430e33666c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:25.469664 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59690f5-edcb-4339-91ac-dc430e33666c None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:25.485344 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-b59690f5-edcb-4339-91ac-dc430e33666c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:17:25.485344 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:17:25.485344 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:17:25.485344 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:17:25.485344 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:17:25.485344 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:17:25.485344 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:17:25.485344 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:17:25.485344 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:17:25.485344 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:17:25.485344 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:17:25.485344 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:17:25.485344 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:17:25.485344 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:17:25.485344 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:17:25.485344 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:17:25.486257 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:17:25.486257 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:17:25.486257 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:17:25.486257 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:17:25.486257 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:17:25.486257 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:17:25.486257 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:17:25.486257 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Nov 02 07:17:25.486257 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:17:25.486257 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:17:25.486257 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:17:25.486257 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:17:25.486257 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:17:25.486257 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:17:25.486257 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:17:25.486257 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:17:25.486257 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:17:25.487189 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:17:25.487189 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:17:25.487189 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Nov 02 07:17:25.490204 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 266/534] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Nov 2 07:17:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:17:25.495039 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05b1f67-dd2a-4ea6-b189-7a14b747083d None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:25.612698 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05b1f67-dd2a-4ea6-b189-7a14b747083d None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:25.624307 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:25.665107 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630b5204-a64f-4ec8-836b-2f718a0e3321 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:25.666141 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630b5204-a64f-4ec8-836b-2f718a0e3321 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:25.666666 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630b5204-a64f-4ec8-836b-2f718a0e3321 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:25.667130 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630b5204-a64f-4ec8-836b-2f718a0e3321 None admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:25.667970 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630b5204-a64f-4ec8-836b-2f718a0e3321 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:17:25.668479 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630b5204-a64f-4ec8-836b-2f718a0e3321 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:25.670018 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630b5204-a64f-4ec8-836b-2f718a0e3321 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:25.683782 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64f8dea-c290-4178-a4c6-1f022c511614 None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:25.684286 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64f8dea-c290-4178-a4c6-1f022c511614 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:25.684286 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64f8dea-c290-4178-a4c6-1f022c511614 None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:25.685743 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 267/535] 10.222.0.18 () {54 vars in 927 bytes} [Wed Nov 2 07:17:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Nov 02 07:17:25.686911 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 269/536] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Nov 2 07:17:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:17:25.691958 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64f8dea-c290-4178-a4c6-1f022c511614 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:25.702202 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a3de02-196d-49ad-a27d-4c7a09f3ba08 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:25.702202 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a3de02-196d-49ad-a27d-4c7a09f3ba08 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:25.702202 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a3de02-196d-49ad-a27d-4c7a09f3ba08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:25.702684 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:25.735384 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-c2a3de02-196d-49ad-a27d-4c7a09f3ba08 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:17:25.736584 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-c2a3de02-196d-49ad-a27d-4c7a09f3ba08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:17:25.820313 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64f8dea-c290-4178-a4c6-1f022c511614 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:25.862091 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e54bf5-a9f3-4ab2-a1c8-467005e93ed2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:25.863137 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e54bf5-a9f3-4ab2-a1c8-467005e93ed2 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:25.863984 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e54bf5-a9f3-4ab2-a1c8-467005e93ed2 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:25.864722 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e54bf5-a9f3-4ab2-a1c8-467005e93ed2 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:25.874816 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e54bf5-a9f3-4ab2-a1c8-467005e93ed2 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:25.876545 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e54bf5-a9f3-4ab2-a1c8-467005e93ed2 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:25.890577 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-92e54bf5-a9f3-4ab2-a1c8-467005e93ed2 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Nov 02 07:17:25.890577 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:17:25.890577 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:17:25.890577 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:17:25.890577 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:17:25.890577 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:17:25.890577 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:17:25.890577 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:17:25.890577 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:17:25.890577 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:17:25.890577 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:17:25.890577 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:17:25.890577 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Nov 02 07:17:25.890577 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_user(user_id) Nov 02 07:17:25.890577 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Nov 02 07:17:25.890577 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Nov 02 07:17:25.891543 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:17:25.891543 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:17:25.891543 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Nov 02 07:17:25.891543 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:17:25.891543 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Nov 02 07:17:25.891543 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:17:25.891543 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:17:25.891543 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:17:25.891543 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:17:25.891543 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:17:25.891543 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:17:25.891543 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Nov 02 07:17:25.891543 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:17:25.891543 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:17:25.891543 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:17:25.891543 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:17:25.891543 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:17:25.891543 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:17:25.892481 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:17:25.892481 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:17:25.892481 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Nov 02 07:17:25.892481 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Nov 02 07:17:25.892481 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Nov 02 07:17:25.892481 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Nov 02 07:17:25.892481 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Nov 02 07:17:25.892481 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Nov 02 07:17:25.892481 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Nov 02 07:17:25.892481 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Nov 02 07:17:25.893005 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 270/537] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Nov 2 07:17:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:17:25.895014 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 268/538] 10.222.0.18 () {56 vars in 1093 bytes} [Wed Nov 2 07:17:25 2022] GET /identity/v3/users/nova => generated 82 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:17:25.902441 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a3de02-196d-49ad-a27d-4c7a09f3ba08 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:26.035225 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:26.039700 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a3de02-196d-49ad-a27d-4c7a09f3ba08 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:26.090453 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc734de4-4e10-4526-8048-c50481f2f66d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:26.090990 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc734de4-4e10-4526-8048-c50481f2f66d None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:26.091271 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc734de4-4e10-4526-8048-c50481f2f66d None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:26.091543 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc734de4-4e10-4526-8048-c50481f2f66d None admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:26.092267 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc734de4-4e10-4526-8048-c50481f2f66d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:17:26.092571 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc734de4-4e10-4526-8048-c50481f2f66d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:26.093984 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc734de4-4e10-4526-8048-c50481f2f66d None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:26.118129 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 271/539] 10.222.0.18 () {56 vars in 1133 bytes} [Wed Nov 2 07:17:25 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:17:26.123601 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e54bf5-a9f3-4ab2-a1c8-467005e93ed2 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:26.215661 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:26.248764 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc734de4-4e10-4526-8048-c50481f2f66d None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:26.280258 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e54bf5-a9f3-4ab2-a1c8-467005e93ed2 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:26.323741 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c62671-836a-4af6-becd-9238281951c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:26.324363 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c62671-836a-4af6-becd-9238281951c3 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:26.324641 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c62671-836a-4af6-becd-9238281951c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:26.324906 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c62671-836a-4af6-becd-9238281951c3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:26.361319 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc734de4-4e10-4526-8048-c50481f2f66d None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:26.399059 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32bf863-e6a1-4f1e-99e9-abc089d96666 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:26.400074 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32bf863-e6a1-4f1e-99e9-abc089d96666 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:26.400553 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32bf863-e6a1-4f1e-99e9-abc089d96666 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:26.400884 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c62671-836a-4af6-becd-9238281951c3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:26.401254 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32bf863-e6a1-4f1e-99e9-abc089d96666 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:26.402346 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c62671-836a-4af6-becd-9238281951c3 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:26.412880 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32bf863-e6a1-4f1e-99e9-abc089d96666 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:17:26.418166 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-b32bf863-e6a1-4f1e-99e9-abc089d96666 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Nov 02 07:17:26.418166 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:17:26.418166 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:17:26.418166 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:17:26.418166 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:17:26.418166 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:17:26.418166 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:17:26.418166 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:17:26.418166 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:17:26.418166 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:17:26.418166 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:17:26.418166 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:17:26.418166 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:17:26.418166 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:17:26.418166 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:17:26.418166 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:17:26.419100 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:17:26.419100 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:17:26.419100 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:17:26.419100 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:17:26.419100 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:17:26.419100 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:17:26.419100 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:17:26.419100 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:17:26.419100 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:17:26.419100 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:17:26.419100 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:17:26.419100 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:17:26.419100 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:17:26.419100 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:17:26.419100 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:17:26.419100 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:17:26.419931 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:17:26.419931 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:17:26.419931 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:17:26.419931 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:17:26.419931 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:17:26.419931 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:17:26.419931 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:17:26.419931 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:17:26.419931 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:17:26.419931 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:17:26.419931 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Nov 02 07:17:26.419931 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:17:26.422723 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 272/540] 10.222.0.18 () {56 vars in 1102 bytes} [Wed Nov 2 07:17:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 07:17:26.424234 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9ea4a3-3734-46ee-95a7-fcca2805a47d None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:26.424765 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9ea4a3-3734-46ee-95a7-fcca2805a47d None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:26.425243 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9ea4a3-3734-46ee-95a7-fcca2805a47d None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:26.427761 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 273/541] 10.222.0.18 () {54 vars in 927 bytes} [Wed Nov 2 07:17:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:17:26.428781 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9ea4a3-3734-46ee-95a7-fcca2805a47d None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:26.448596 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:26.473239 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:26.518197 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 269/542] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Nov 2 07:17:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Nov 02 07:17:26.519574 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebc2315-d088-4bb7-a638-f02ba309ce0c None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:26.519867 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebc2315-d088-4bb7-a638-f02ba309ce0c None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:26.520138 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebc2315-d088-4bb7-a638-f02ba309ce0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:26.540168 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-cebc2315-d088-4bb7-a638-f02ba309ce0c None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:17:26.541369 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-cebc2315-d088-4bb7-a638-f02ba309ce0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:17:26.544968 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9ea4a3-3734-46ee-95a7-fcca2805a47d None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:26.583081 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca797e5-0a00-4307-894e-1e1cc0ecd15f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:26.584076 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca797e5-0a00-4307-894e-1e1cc0ecd15f None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:26.584568 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca797e5-0a00-4307-894e-1e1cc0ecd15f None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:26.585031 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca797e5-0a00-4307-894e-1e1cc0ecd15f None admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:26.585875 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca797e5-0a00-4307-894e-1e1cc0ecd15f None admin] RBAC: Adding query filter params (name=service) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:17:26.586406 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca797e5-0a00-4307-894e-1e1cc0ecd15f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:26.588013 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca797e5-0a00-4307-894e-1e1cc0ecd15f None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:26.597100 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 274/543] 10.222.0.18 () {56 vars in 1103 bytes} [Wed Nov 2 07:17:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:17:26.598301 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca797e5-0a00-4307-894e-1e1cc0ecd15f None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:26.622038 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 270/544] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Nov 2 07:17:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:17:26.622888 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebc2315-d088-4bb7-a638-f02ba309ce0c None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:26.697450 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:26.731385 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca797e5-0a00-4307-894e-1e1cc0ecd15f None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:26.797635 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fdde12-5c05-455b-b59e-7c255c5243e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:26.798722 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fdde12-5c05-455b-b59e-7c255c5243e2 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:26.799235 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fdde12-5c05-455b-b59e-7c255c5243e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:26.799726 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fdde12-5c05-455b-b59e-7c255c5243e2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:26.812958 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fdde12-5c05-455b-b59e-7c255c5243e2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:26.814861 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fdde12-5c05-455b-b59e-7c255c5243e2 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:26.826398 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:26.844576 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-f3fdde12-5c05-455b-b59e-7c255c5243e2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:17:26.844576 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:17:26.844576 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:17:26.844576 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:17:26.844576 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:17:26.844576 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:17:26.844576 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:17:26.844576 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:17:26.844576 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:17:26.844576 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:17:26.844576 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:17:26.844576 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:17:26.844576 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:17:26.844576 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:17:26.844576 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:17:26.844576 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:17:26.845595 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:17:26.845595 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:17:26.845595 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:17:26.845595 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:17:26.845595 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:17:26.845595 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:17:26.845595 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:17:26.845595 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:17:26.845595 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:17:26.845595 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:17:26.845595 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:17:26.845595 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:17:26.845595 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:17:26.845595 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:17:26.845595 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:17:26.845595 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:17:26.845595 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:17:26.846512 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:17:26.846512 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:17:26.846512 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:17:26.846682 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebc2315-d088-4bb7-a638-f02ba309ce0c None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:26.851081 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 275/545] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Nov 2 07:17:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:17:26.855645 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fdde12-5c05-455b-b59e-7c255c5243e2 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:26.919186 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a279414c-d2d0-450f-b3c5-41cac3c7277c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:26.920158 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a279414c-d2d0-450f-b3c5-41cac3c7277c None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:26.920641 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a279414c-d2d0-450f-b3c5-41cac3c7277c None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:26.921111 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a279414c-d2d0-450f-b3c5-41cac3c7277c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:26.935701 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a279414c-d2d0-450f-b3c5-41cac3c7277c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:26.938213 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a279414c-d2d0-450f-b3c5-41cac3c7277c None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:26.956912 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-a279414c-d2d0-450f-b3c5-41cac3c7277c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:17:26.956912 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:17:26.956912 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:17:26.956912 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:17:26.956912 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:17:26.956912 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:17:26.956912 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:17:26.956912 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:17:26.956912 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:17:26.956912 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:17:26.956912 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:17:26.956912 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:17:26.956912 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:17:26.956912 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:17:26.956912 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:17:26.956912 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:17:26.959093 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:17:26.959093 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:17:26.959093 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:17:26.959093 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:17:26.959093 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:17:26.959093 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:17:26.959093 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:17:26.959093 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Nov 02 07:17:26.959093 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:17:26.959093 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:17:26.959093 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:17:26.959093 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:17:26.959093 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:17:26.959093 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:17:26.959093 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:17:26.959093 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:17:26.959093 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:17:26.960434 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:17:26.960434 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:17:26.960434 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Nov 02 07:17:26.960434 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 271/546] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Nov 2 07:17:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 337 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:17:26.961232 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71aa1028-17c0-454b-b226-78fd91d2d108 None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:26.961521 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71aa1028-17c0-454b-b226-78fd91d2d108 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:26.961794 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71aa1028-17c0-454b-b226-78fd91d2d108 None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:26.963868 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 272/547] 10.222.0.18 () {54 vars in 927 bytes} [Wed Nov 2 07:17:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:17:26.964907 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f17092-1da4-409e-9614-3e054814f3a6 None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:26.965213 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f17092-1da4-409e-9614-3e054814f3a6 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:26.965447 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f17092-1da4-409e-9614-3e054814f3a6 None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:26.970484 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 273/548] 10.222.0.18 () {54 vars in 927 bytes} [Wed Nov 2 07:17:26 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:17:26.971569 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f17092-1da4-409e-9614-3e054814f3a6 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:26.997463 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fdde12-5c05-455b-b59e-7c255c5243e2 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:27.027196 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:27.033264 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8187b067-d685-46c7-bdda-9d1341a4edbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:27.033865 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8187b067-d685-46c7-bdda-9d1341a4edbf None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:27.034141 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8187b067-d685-46c7-bdda-9d1341a4edbf None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:27.034446 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8187b067-d685-46c7-bdda-9d1341a4edbf None admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:27.035083 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8187b067-d685-46c7-bdda-9d1341a4edbf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:17:27.035383 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8187b067-d685-46c7-bdda-9d1341a4edbf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:27.036736 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8187b067-d685-46c7-bdda-9d1341a4edbf None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:27.048034 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 276/549] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Nov 2 07:17:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:17:27.049175 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654ee851-ef03-473b-b11c-be0a4bbd6d0b None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:27.049455 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654ee851-ef03-473b-b11c-be0a4bbd6d0b None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:27.049725 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654ee851-ef03-473b-b11c-be0a4bbd6d0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:27.067914 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-654ee851-ef03-473b-b11c-be0a4bbd6d0b None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:17:27.069276 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-654ee851-ef03-473b-b11c-be0a4bbd6d0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:17:27.078601 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f17092-1da4-409e-9614-3e054814f3a6 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:27.111727 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07878035-8e4d-4179-9b28-a39e2a3591cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:27.112359 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07878035-8e4d-4179-9b28-a39e2a3591cd None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:27.112643 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07878035-8e4d-4179-9b28-a39e2a3591cd None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:27.112910 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07878035-8e4d-4179-9b28-a39e2a3591cd None admin] PATH_INFO: `/v3/domains` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:27.113628 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07878035-8e4d-4179-9b28-a39e2a3591cd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:17:27.113826 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07878035-8e4d-4179-9b28-a39e2a3591cd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:27.116044 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07878035-8e4d-4179-9b28-a39e2a3591cd None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:27.127541 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 274/550] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Nov 2 07:17:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:17:27.128723 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4d83d5-6703-42b8-9a75-2d14cea90025 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:27.129008 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4d83d5-6703-42b8-9a75-2d14cea90025 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:27.129280 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4d83d5-6703-42b8-9a75-2d14cea90025 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:27.134921 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 277/551] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Nov 2 07:17:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:17:27.135722 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654ee851-ef03-473b-b11c-be0a4bbd6d0b None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:27.143589 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:27.154590 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-bd4d83d5-6703-42b8-9a75-2d14cea90025 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:17:27.155787 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-bd4d83d5-6703-42b8-9a75-2d14cea90025 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:17:27.225903 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 275/552] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Nov 2 07:17:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:17:27.227027 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4d83d5-6703-42b8-9a75-2d14cea90025 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:27.260479 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654ee851-ef03-473b-b11c-be0a4bbd6d0b None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:27.287427 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:27.307222 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2ab1cd-fd67-4439-b303-16b95133603c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:27.308072 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2ab1cd-fd67-4439-b303-16b95133603c None admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:27.308497 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2ab1cd-fd67-4439-b303-16b95133603c None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:27.308913 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2ab1cd-fd67-4439-b303-16b95133603c None admin] PATH_INFO: `/v3/services` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:27.310059 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2ab1cd-fd67-4439-b303-16b95133603c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:27.312184 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2ab1cd-fd67-4439-b303-16b95133603c None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:27.342781 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 278/553] 10.222.0.18 () {60 vars in 1141 bytes} [Wed Nov 2 07:17:27 2022] POST /identity/v3/services => generated 251 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:17:27.342781 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2ab1cd-fd67-4439-b303-16b95133603c None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:27.347315 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4d83d5-6703-42b8-9a75-2d14cea90025 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:27.419285 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:27.425447 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59327c9-f2e9-4374-8658-6534ee43d9f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:27.426206 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59327c9-f2e9-4374-8658-6534ee43d9f2 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:27.426644 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59327c9-f2e9-4374-8658-6534ee43d9f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:27.427006 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59327c9-f2e9-4374-8658-6534ee43d9f2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:27.436546 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59327c9-f2e9-4374-8658-6534ee43d9f2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:27.438073 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59327c9-f2e9-4374-8658-6534ee43d9f2 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:27.452053 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-a59327c9-f2e9-4374-8658-6534ee43d9f2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 07:17:27.452053 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:17:27.452053 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:17:27.452053 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:17:27.452053 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:17:27.452053 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:17:27.452053 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:17:27.452053 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:17:27.452053 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:17:27.452053 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:17:27.452053 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:17:27.452053 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:17:27.452053 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Nov 02 07:17:27.452053 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_project(project_id) Nov 02 07:17:27.452053 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Nov 02 07:17:27.453028 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Nov 02 07:17:27.453028 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:17:27.453028 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:17:27.453028 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:17:27.453028 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:17:27.453028 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:17:27.453028 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:17:27.453028 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:17:27.453028 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Nov 02 07:17:27.453028 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:17:27.453028 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:17:27.453028 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:17:27.453028 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:17:27.453028 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:17:27.453028 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:17:27.453028 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:17:27.453028 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:17:27.453905 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Nov 02 07:17:27.453905 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Nov 02 07:17:27.453905 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Nov 02 07:17:27.453905 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Nov 02 07:17:27.453905 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Nov 02 07:17:27.453905 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Nov 02 07:17:27.453905 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 07:17:27.453905 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Nov 02 07:17:27.458178 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 276/554] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Nov 2 07:17:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:17:27.459252 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59327c9-f2e9-4374-8658-6534ee43d9f2 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:27.504200 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2ab1cd-fd67-4439-b303-16b95133603c None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:27.541974 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0649c40-ae57-4d8b-8fd5-25da20f3e3cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:27.542636 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0649c40-ae57-4d8b-8fd5-25da20f3e3cb None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:27.542919 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0649c40-ae57-4d8b-8fd5-25da20f3e3cb None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:27.543187 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0649c40-ae57-4d8b-8fd5-25da20f3e3cb None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:27.550903 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0649c40-ae57-4d8b-8fd5-25da20f3e3cb None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:27.552388 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0649c40-ae57-4d8b-8fd5-25da20f3e3cb None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:27.564290 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-f0649c40-ae57-4d8b-8fd5-25da20f3e3cb None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Nov 02 07:17:27.564290 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:17:27.564290 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:17:27.564290 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:17:27.564290 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:17:27.564290 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:17:27.564290 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:17:27.564290 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:17:27.564290 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:17:27.564290 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:17:27.564290 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:17:27.564290 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:17:27.564290 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Nov 02 07:17:27.564290 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_user(user_id) Nov 02 07:17:27.564290 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Nov 02 07:17:27.564290 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Nov 02 07:17:27.565239 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:17:27.565239 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:17:27.565239 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Nov 02 07:17:27.565239 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:17:27.565239 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Nov 02 07:17:27.565239 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:17:27.565239 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:17:27.565239 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:17:27.565239 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:17:27.565239 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:17:27.565239 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:17:27.565239 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:17:27.565239 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:17:27.565239 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:17:27.565239 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:17:27.565239 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:17:27.565239 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:17:27.565239 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:17:27.566171 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:17:27.566171 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:17:27.566171 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Nov 02 07:17:27.566171 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Nov 02 07:17:27.566171 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Nov 02 07:17:27.566171 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Nov 02 07:17:27.566171 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Nov 02 07:17:27.566171 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Nov 02 07:17:27.566171 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Nov 02 07:17:27.566171 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:17:27.566171 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 279/555] 10.222.0.18 () {56 vars in 1117 bytes} [Wed Nov 2 07:17:27 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:17:27.566171 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0649c40-ae57-4d8b-8fd5-25da20f3e3cb None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:27.583950 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59327c9-f2e9-4374-8658-6534ee43d9f2 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:27.617243 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:27.618052 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd01e9f7-b481-4edc-af8e-c38b2887bc7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:27.618851 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd01e9f7-b481-4edc-af8e-c38b2887bc7d None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:27.619133 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd01e9f7-b481-4edc-af8e-c38b2887bc7d None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:27.619408 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd01e9f7-b481-4edc-af8e-c38b2887bc7d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:27.631988 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd01e9f7-b481-4edc-af8e-c38b2887bc7d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:17:27.638768 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-cd01e9f7-b481-4edc-af8e-c38b2887bc7d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 07:17:27.638768 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:17:27.638768 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:17:27.638768 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:17:27.638768 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:17:27.638768 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:17:27.638768 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:17:27.638768 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:17:27.638768 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:17:27.638768 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:17:27.638768 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:17:27.638768 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:17:27.638768 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:17:27.638768 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:17:27.638768 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:17:27.638768 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:17:27.639711 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:17:27.639711 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:17:27.639711 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:17:27.639711 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:17:27.639711 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:17:27.639711 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:17:27.639711 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:17:27.639711 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:17:27.639711 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:17:27.639711 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:17:27.639711 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:17:27.639711 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Nov 02 07:17:27.639711 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:17:27.639711 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:17:27.639711 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:17:27.639711 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:17:27.640591 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:17:27.640591 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:17:27.640591 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:17:27.640591 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:17:27.640591 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:17:27.640591 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:17:27.640591 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:17:27.640591 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:17:27.640591 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:17:27.640591 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:17:27.640591 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 07:17:27.640591 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Nov 02 07:17:27.640591 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 277/556] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Nov 2 07:17:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:17:27.640591 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd01e9f7-b481-4edc-af8e-c38b2887bc7d None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:27.688368 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0649c40-ae57-4d8b-8fd5-25da20f3e3cb None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:27.700675 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:27.724800 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5397d743-74a6-4a10-beb5-6b53fbf34c8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:27.725451 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5397d743-74a6-4a10-beb5-6b53fbf34c8d None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:27.725674 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5397d743-74a6-4a10-beb5-6b53fbf34c8d None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:27.725941 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5397d743-74a6-4a10-beb5-6b53fbf34c8d None admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:27.726660 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5397d743-74a6-4a10-beb5-6b53fbf34c8d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:17:27.726960 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5397d743-74a6-4a10-beb5-6b53fbf34c8d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:27.728679 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5397d743-74a6-4a10-beb5-6b53fbf34c8d None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:27.741073 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 280/557] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Nov 2 07:17:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:17:27.746567 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5397d743-74a6-4a10-beb5-6b53fbf34c8d None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:27.754660 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd01e9f7-b481-4edc-af8e-c38b2887bc7d None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:27.792675 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761da87d-257d-4c74-b31f-123b664a8430 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:27.793289 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761da87d-257d-4c74-b31f-123b664a8430 None admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:27.793567 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761da87d-257d-4c74-b31f-123b664a8430 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:27.793837 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761da87d-257d-4c74-b31f-123b664a8430 None admin] PATH_INFO: `/v3/services` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:27.794532 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761da87d-257d-4c74-b31f-123b664a8430 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:27.795890 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761da87d-257d-4c74-b31f-123b664a8430 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:27.812657 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 278/558] 10.222.0.18 () {60 vars in 1141 bytes} [Wed Nov 2 07:17:27 2022] POST /identity/v3/services => generated 240 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:17:27.814271 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761da87d-257d-4c74-b31f-123b664a8430 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:27.860254 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:27.864697 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5397d743-74a6-4a10-beb5-6b53fbf34c8d None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:27.915484 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8f8bc6-b857-4976-a964-89d8b9795749 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:27.916467 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8f8bc6-b857-4976-a964-89d8b9795749 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:27.916942 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8f8bc6-b857-4976-a964-89d8b9795749 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:27.917433 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8f8bc6-b857-4976-a964-89d8b9795749 None admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:27.918382 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8f8bc6-b857-4976-a964-89d8b9795749 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:17:27.918887 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8f8bc6-b857-4976-a964-89d8b9795749 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:27.920554 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8f8bc6-b857-4976-a964-89d8b9795749 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:27.941807 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 281/559] 10.222.0.18 () {56 vars in 1149 bytes} [Wed Nov 2 07:17:27 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:17:27.942590 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8f8bc6-b857-4976-a964-89d8b9795749 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:27.966870 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761da87d-257d-4c74-b31f-123b664a8430 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:27.980145 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:28.007084 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f28674-5bdc-4a7f-8467-b2fe5c1b375f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:28.008012 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f28674-5bdc-4a7f-8467-b2fe5c1b375f None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:28.008516 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f28674-5bdc-4a7f-8467-b2fe5c1b375f None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:28.008786 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f28674-5bdc-4a7f-8467-b2fe5c1b375f None admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:28.009447 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f28674-5bdc-4a7f-8467-b2fe5c1b375f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:17:28.009782 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f28674-5bdc-4a7f-8467-b2fe5c1b375f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:28.011217 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f28674-5bdc-4a7f-8467-b2fe5c1b375f None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:28.021101 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 279/560] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Nov 2 07:17:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:17:28.022585 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f28674-5bdc-4a7f-8467-b2fe5c1b375f None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:28.075895 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8f8bc6-b857-4976-a964-89d8b9795749 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:28.121364 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bae746-1af7-4af2-b42f-58becb63fd41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:28.122062 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bae746-1af7-4af2-b42f-58becb63fd41 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:28.122343 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bae746-1af7-4af2-b42f-58becb63fd41 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:28.122638 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bae746-1af7-4af2-b42f-58becb63fd41 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:28.123463 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bae746-1af7-4af2-b42f-58becb63fd41 None admin] RBAC: Adding query filter params (role.id=c4bf019fc6c44c71a6c984f6d0bab75f, scope.project.id=af04daa468b64160a6176dcab29502a7, user.id=69e48cba3f804cf08b2477f01ae95179) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:17:28.123769 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bae746-1af7-4af2-b42f-58becb63fd41 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:28.125250 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bae746-1af7-4af2-b42f-58becb63fd41 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:28.132846 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 282/561] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Nov 2 07:17:27 2022] GET /identity/v3/role_assignments?user.id=69e48cba3f804cf08b2477f01ae95179&scope.project.id=af04daa468b64160a6176dcab29502a7&role.id=c4bf019fc6c44c71a6c984f6d0bab75f => generated 608 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:17:28.133477 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bae746-1af7-4af2-b42f-58becb63fd41 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:28.138321 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f28674-5bdc-4a7f-8467-b2fe5c1b375f None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:28.176459 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:28.183108 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b90e083-65fe-4dde-94d7-8363f53682ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:28.184133 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b90e083-65fe-4dde-94d7-8363f53682ef None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:28.184611 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b90e083-65fe-4dde-94d7-8363f53682ef None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:28.185158 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b90e083-65fe-4dde-94d7-8363f53682ef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:28.200723 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b90e083-65fe-4dde-94d7-8363f53682ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:28.202833 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b90e083-65fe-4dde-94d7-8363f53682ef None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:28.222281 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-9b90e083-65fe-4dde-94d7-8363f53682ef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:17:28.222281 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:17:28.222281 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:17:28.222281 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:17:28.222281 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:17:28.222281 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:17:28.222281 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:17:28.222281 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:17:28.222281 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:17:28.222281 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:17:28.222281 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:17:28.222281 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:17:28.222281 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:17:28.222281 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:17:28.222281 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:17:28.222281 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:17:28.224017 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:17:28.224017 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:17:28.224017 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:17:28.224017 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:17:28.224017 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:17:28.224017 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:17:28.224017 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:17:28.224017 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Nov 02 07:17:28.224017 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:17:28.224017 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:17:28.224017 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:17:28.224017 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:17:28.224017 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:17:28.224017 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:17:28.224017 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:17:28.224017 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:17:28.224017 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:17:28.224997 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:17:28.224997 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:17:28.224997 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Nov 02 07:17:28.228417 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 280/562] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Nov 2 07:17:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:17:28.233461 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b90e083-65fe-4dde-94d7-8363f53682ef None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:28.283512 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:28.356011 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bae746-1af7-4af2-b42f-58becb63fd41 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:28.422870 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b90e083-65fe-4dde-94d7-8363f53682ef None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:28.427015 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1769ea22-3cf8-4f7a-aeed-3397ff3b5661 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:28.427771 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1769ea22-3cf8-4f7a-aeed-3397ff3b5661 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:28.428124 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1769ea22-3cf8-4f7a-aeed-3397ff3b5661 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:28.428468 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1769ea22-3cf8-4f7a-aeed-3397ff3b5661 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:28.441270 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1769ea22-3cf8-4f7a-aeed-3397ff3b5661 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:28.442979 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1769ea22-3cf8-4f7a-aeed-3397ff3b5661 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:28.460534 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-1769ea22-3cf8-4f7a-aeed-3397ff3b5661 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:17:28.460534 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:17:28.460534 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:17:28.460534 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:17:28.460534 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:17:28.460534 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:17:28.460534 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:17:28.460534 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:17:28.460534 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:17:28.460534 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:17:28.460534 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:17:28.460534 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:17:28.460534 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:17:28.460534 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:17:28.460534 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:17:28.460534 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:17:28.461513 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:17:28.461513 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:17:28.461513 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:17:28.461513 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:17:28.461513 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:17:28.461513 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:17:28.461513 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:17:28.461513 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:17:28.461513 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:17:28.461513 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:17:28.461513 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:17:28.461513 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:17:28.461513 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:17:28.461513 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:17:28.461513 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:17:28.461513 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:17:28.461513 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:17:28.462420 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:17:28.462420 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:17:28.462420 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:17:28.462420 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 283/563] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Nov 2 07:17:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 328 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:17:28.471063 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570df596-fe28-4ba6-970e-653489a083a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:28.471063 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570df596-fe28-4ba6-970e-653489a083a0 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:28.471063 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570df596-fe28-4ba6-970e-653489a083a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:28.471063 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570df596-fe28-4ba6-970e-653489a083a0 None admin] PATH_INFO: `/v3/domains` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:28.471410 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570df596-fe28-4ba6-970e-653489a083a0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:17:28.471624 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570df596-fe28-4ba6-970e-653489a083a0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:28.473141 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1769ea22-3cf8-4f7a-aeed-3397ff3b5661 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:28.474291 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570df596-fe28-4ba6-970e-653489a083a0 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:28.491636 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 281/564] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Nov 2 07:17:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:17:28.499446 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570df596-fe28-4ba6-970e-653489a083a0 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:28.551054 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:28.587090 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1769ea22-3cf8-4f7a-aeed-3397ff3b5661 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:28.624414 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cf74fe-2b23-48e9-91c1-4a02423f79fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:28.625171 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cf74fe-2b23-48e9-91c1-4a02423f79fa None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:28.625554 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cf74fe-2b23-48e9-91c1-4a02423f79fa None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:28.625929 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cf74fe-2b23-48e9-91c1-4a02423f79fa None admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:28.626718 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63cf74fe-2b23-48e9-91c1-4a02423f79fa None admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:17:28.627587 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63cf74fe-2b23-48e9-91c1-4a02423f79fa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:28.629190 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63cf74fe-2b23-48e9-91c1-4a02423f79fa None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:28.643827 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 284/565] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Nov 2 07:17:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:17:28.652074 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cf74fe-2b23-48e9-91c1-4a02423f79fa None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:28.702278 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:28.747415 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570df596-fe28-4ba6-970e-653489a083a0 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:28.770726 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cf74fe-2b23-48e9-91c1-4a02423f79fa None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:28.805247 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:28.812728 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45168d99-5fd2-4c7e-9896-b84ace0ea82f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:28.813440 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45168d99-5fd2-4c7e-9896-b84ace0ea82f None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:28.813968 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45168d99-5fd2-4c7e-9896-b84ace0ea82f None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:28.814603 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45168d99-5fd2-4c7e-9896-b84ace0ea82f None admin] PATH_INFO: `/v3/users/swift` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:28.823637 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45168d99-5fd2-4c7e-9896-b84ace0ea82f None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:28.825161 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45168d99-5fd2-4c7e-9896-b84ace0ea82f None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:28.835066 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbec82b-731c-4645-b7ef-ce17b25b5247 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:28.835799 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbec82b-731c-4645-b7ef-ce17b25b5247 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:28.836080 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbec82b-731c-4645-b7ef-ce17b25b5247 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:28.836352 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbec82b-731c-4645-b7ef-ce17b25b5247 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:28.837633 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-45168d99-5fd2-4c7e-9896-b84ace0ea82f None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Nov 02 07:17:28.837633 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:17:28.837633 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:17:28.837633 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:17:28.837633 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:17:28.837633 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:17:28.837633 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:17:28.837633 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:17:28.837633 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:17:28.837633 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:17:28.837633 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:17:28.837633 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:17:28.837633 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Nov 02 07:17:28.837633 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_user(user_id) Nov 02 07:17:28.837633 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Nov 02 07:17:28.837633 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Nov 02 07:17:28.839759 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:17:28.839759 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:17:28.839759 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Nov 02 07:17:28.839759 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:17:28.839759 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Nov 02 07:17:28.839759 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:17:28.839759 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:17:28.839759 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:17:28.839759 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:17:28.839759 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:17:28.839759 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:17:28.839759 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:17:28.839759 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:17:28.839759 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:17:28.839759 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:17:28.839759 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:17:28.839759 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:17:28.839759 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:17:28.841901 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:17:28.841901 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:17:28.841901 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Nov 02 07:17:28.841901 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Nov 02 07:17:28.841901 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Nov 02 07:17:28.841901 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Nov 02 07:17:28.841901 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Nov 02 07:17:28.841901 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Nov 02 07:17:28.841901 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Nov 02 07:17:28.841901 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:17:28.841901 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 285/566] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Nov 2 07:17:28 2022] GET /identity/v3/users/swift => generated 83 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 07:17:28.848044 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbec82b-731c-4645-b7ef-ce17b25b5247 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:28.850181 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbec82b-731c-4645-b7ef-ce17b25b5247 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:28.851451 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45168d99-5fd2-4c7e-9896-b84ace0ea82f None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:28.864279 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-8fbec82b-731c-4645-b7ef-ce17b25b5247 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 07:17:28.864279 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:17:28.864279 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:17:28.864279 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:17:28.864279 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:17:28.864279 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:17:28.864279 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:17:28.864279 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:17:28.864279 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:17:28.864279 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:17:28.864279 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:17:28.864279 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:17:28.864279 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Nov 02 07:17:28.864279 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_project(project_id) Nov 02 07:17:28.864279 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Nov 02 07:17:28.865230 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Nov 02 07:17:28.865230 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:17:28.865230 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:17:28.865230 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:17:28.865230 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:17:28.865230 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:17:28.865230 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:17:28.865230 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:17:28.865230 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Nov 02 07:17:28.865230 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:17:28.865230 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:17:28.865230 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:17:28.865230 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:17:28.865230 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:17:28.865230 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:17:28.865230 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:17:28.865230 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:17:28.866208 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Nov 02 07:17:28.866208 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Nov 02 07:17:28.866208 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Nov 02 07:17:28.866208 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Nov 02 07:17:28.866208 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Nov 02 07:17:28.866208 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Nov 02 07:17:28.866208 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 07:17:28.866208 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Nov 02 07:17:28.868711 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 282/567] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Nov 2 07:17:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 370 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:17:28.874647 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbec82b-731c-4645-b7ef-ce17b25b5247 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:29.052753 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45168d99-5fd2-4c7e-9896-b84ace0ea82f None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:29.056287 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbec82b-731c-4645-b7ef-ce17b25b5247 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:29.108871 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa3f89b-5cf6-4174-974d-e4eb291cb447 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:29.109483 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa3f89b-5cf6-4174-974d-e4eb291cb447 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:29.109765 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa3f89b-5cf6-4174-974d-e4eb291cb447 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:29.110037 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa3f89b-5cf6-4174-974d-e4eb291cb447 None admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:29.111631 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa3f89b-5cf6-4174-974d-e4eb291cb447 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:17:29.111944 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa3f89b-5cf6-4174-974d-e4eb291cb447 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:29.114160 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa3f89b-5cf6-4174-974d-e4eb291cb447 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:29.124081 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3655c200-944d-4bc6-a268-5c167cfe2bb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:29.124691 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3655c200-944d-4bc6-a268-5c167cfe2bb9 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:29.124971 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3655c200-944d-4bc6-a268-5c167cfe2bb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:29.125256 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3655c200-944d-4bc6-a268-5c167cfe2bb9 None admin] PATH_INFO: `/v3/projects` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:29.125930 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3655c200-944d-4bc6-a268-5c167cfe2bb9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:17:29.126237 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3655c200-944d-4bc6-a268-5c167cfe2bb9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:29.132264 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3655c200-944d-4bc6-a268-5c167cfe2bb9 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:29.151662 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 286/568] 10.222.0.18 () {56 vars in 1135 bytes} [Wed Nov 2 07:17:28 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:17:29.159597 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa3f89b-5cf6-4174-974d-e4eb291cb447 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:29.166930 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 283/569] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Nov 2 07:17:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:17:29.168030 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3655c200-944d-4bc6-a268-5c167cfe2bb9 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:29.182716 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:29.261866 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:29.328212 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3655c200-944d-4bc6-a268-5c167cfe2bb9 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:29.368431 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cb9f5a-5cd3-4c9d-a302-ec262d9246b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:29.369089 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cb9f5a-5cd3-4c9d-a302-ec262d9246b5 None admin] REQUEST_METHOD: `PUT` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:29.369424 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cb9f5a-5cd3-4c9d-a302-ec262d9246b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:29.369698 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cb9f5a-5cd3-4c9d-a302-ec262d9246b5 None admin] PATH_INFO: `/v3/projects/af04daa468b64160a6176dcab29502a7/users/ff4736f3d50746b0bf5d6a917b2ca0cf/roles/4af6f4c514284b8cbddf3e02ec826378` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:29.372287 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa3f89b-5cf6-4174-974d-e4eb291cb447 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:29.385484 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cb9f5a-5cd3-4c9d-a302-ec262d9246b5 None admin] RBAC: Authorizing `identity:create_grant(project_id=af04daa468b64160a6176dcab29502a7, user_id=ff4736f3d50746b0bf5d6a917b2ca0cf, role_id=4af6f4c514284b8cbddf3e02ec826378)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:29.386945 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cb9f5a-5cd3-4c9d-a302-ec262d9246b5 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:29.403060 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 284/570] 10.222.0.18 () {58 vars in 1439 bytes} [Wed Nov 2 07:17:29 2022] PUT /identity/v3/projects/af04daa468b64160a6176dcab29502a7/users/ff4736f3d50746b0bf5d6a917b2ca0cf/roles/4af6f4c514284b8cbddf3e02ec826378 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 07:17:29.450414 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8baffeac-59ed-4d59-bd8b-74a82275ef94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:29.451073 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baffeac-59ed-4d59-bd8b-74a82275ef94 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:29.451351 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baffeac-59ed-4d59-bd8b-74a82275ef94 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:29.451619 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baffeac-59ed-4d59-bd8b-74a82275ef94 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:29.466630 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8baffeac-59ed-4d59-bd8b-74a82275ef94 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:29.468098 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8baffeac-59ed-4d59-bd8b-74a82275ef94 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:29.493176 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-8baffeac-59ed-4d59-bd8b-74a82275ef94 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:17:29.493176 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:17:29.493176 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:17:29.493176 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:17:29.493176 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:17:29.493176 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:17:29.493176 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:17:29.493176 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:17:29.493176 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:17:29.493176 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:17:29.493176 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:17:29.493176 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:17:29.493176 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:17:29.493176 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:17:29.493176 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:17:29.493176 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:17:29.494246 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:17:29.494246 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:17:29.494246 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:17:29.494246 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:17:29.494246 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:17:29.494246 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:17:29.494246 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:17:29.494246 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:17:29.494246 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:17:29.494246 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:17:29.494246 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:17:29.494246 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:17:29.494246 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:17:29.494246 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:17:29.494246 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:17:29.494246 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:17:29.494246 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:17:29.495162 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:17:29.495162 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:17:29.495162 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:17:29.495162 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 287/571] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Nov 2 07:17:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 336 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:17:29.503619 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cb9f5a-5cd3-4c9d-a302-ec262d9246b5 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:29.602865 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:29.669508 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cb9f5a-5cd3-4c9d-a302-ec262d9246b5 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:29.706256 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1bf84d-2c1f-4ca0-8e4e-89fba8a5621d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:29.706896 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1bf84d-2c1f-4ca0-8e4e-89fba8a5621d None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:29.707180 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1bf84d-2c1f-4ca0-8e4e-89fba8a5621d None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:29.707457 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1bf84d-2c1f-4ca0-8e4e-89fba8a5621d None admin] PATH_INFO: `/v3/domains` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:29.708970 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1bf84d-2c1f-4ca0-8e4e-89fba8a5621d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:17:29.709280 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1bf84d-2c1f-4ca0-8e4e-89fba8a5621d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:29.710712 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1bf84d-2c1f-4ca0-8e4e-89fba8a5621d None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:29.725315 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 285/572] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Nov 2 07:17:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:17:29.730728 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8baffeac-59ed-4d59-bd8b-74a82275ef94 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:29.736551 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b02f51-61f8-4c79-b8d9-b99cd08f98ce None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:29.736840 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b02f51-61f8-4c79-b8d9-b99cd08f98ce None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:29.737111 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b02f51-61f8-4c79-b8d9-b99cd08f98ce None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:29.738702 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 286/573] 10.222.0.18 () {54 vars in 927 bytes} [Wed Nov 2 07:17:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Nov 02 07:17:29.749704 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516bc295-625b-48c0-b367-e2434b58b5de None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:29.750119 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516bc295-625b-48c0-b367-e2434b58b5de None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:29.750387 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516bc295-625b-48c0-b367-e2434b58b5de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:29.779415 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-516bc295-625b-48c0-b367-e2434b58b5de None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:17:29.780609 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-516bc295-625b-48c0-b367-e2434b58b5de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:17:29.890311 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:29.896016 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8baffeac-59ed-4d59-bd8b-74a82275ef94 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:29.917107 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 287/574] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Nov 2 07:17:29 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Nov 02 07:17:29.942510 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb0ca99-66de-4db1-a605-a889b8a7a08a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:29.943143 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb0ca99-66de-4db1-a605-a889b8a7a08a None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:29.943419 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb0ca99-66de-4db1-a605-a889b8a7a08a None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:29.943689 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb0ca99-66de-4db1-a605-a889b8a7a08a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:29.957091 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb0ca99-66de-4db1-a605-a889b8a7a08a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:29.959314 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb0ca99-66de-4db1-a605-a889b8a7a08a None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:29.975682 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-bfb0ca99-66de-4db1-a605-a889b8a7a08a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 07:17:29.975682 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:17:29.975682 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:17:29.975682 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:17:29.975682 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:17:29.975682 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:17:29.975682 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:17:29.975682 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:17:29.975682 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:17:29.975682 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:17:29.975682 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:17:29.975682 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:17:29.975682 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Nov 02 07:17:29.975682 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_project(project_id) Nov 02 07:17:29.975682 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Nov 02 07:17:29.976619 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Nov 02 07:17:29.976619 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:17:29.976619 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:17:29.976619 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:17:29.976619 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:17:29.976619 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:17:29.976619 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:17:29.976619 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:17:29.976619 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:17:29.976619 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:17:29.976619 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:17:29.976619 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:17:29.976619 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:17:29.976619 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:17:29.976619 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:17:29.976619 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:17:29.976619 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:17:29.977498 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Nov 02 07:17:29.977498 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Nov 02 07:17:29.977498 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Nov 02 07:17:29.977498 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Nov 02 07:17:29.977498 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Nov 02 07:17:29.977498 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Nov 02 07:17:29.977498 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 07:17:29.977498 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:17:29.977498 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 288/575] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Nov 2 07:17:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 07:17:29.982711 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516bc295-625b-48c0-b367-e2434b58b5de None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:30.034310 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:30.091718 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516bc295-625b-48c0-b367-e2434b58b5de None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:30.133957 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80f8857-d442-413b-a04d-08e00c8e972a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:30.134650 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80f8857-d442-413b-a04d-08e00c8e972a None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:30.134935 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80f8857-d442-413b-a04d-08e00c8e972a None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:30.135203 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80f8857-d442-413b-a04d-08e00c8e972a None admin] PATH_INFO: `/v3/projects` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:30.135884 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80f8857-d442-413b-a04d-08e00c8e972a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:17:30.136183 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80f8857-d442-413b-a04d-08e00c8e972a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:30.137616 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80f8857-d442-413b-a04d-08e00c8e972a None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:30.150816 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 288/576] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Nov 2 07:17:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:17:30.159429 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb0ca99-66de-4db1-a605-a889b8a7a08a None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:30.282475 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb0ca99-66de-4db1-a605-a889b8a7a08a None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:30.309443 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80f8857-d442-413b-a04d-08e00c8e972a None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:30.325911 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7351595f-d112-431e-989c-a7039655563f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:30.326640 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7351595f-d112-431e-989c-a7039655563f None admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:30.326927 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7351595f-d112-431e-989c-a7039655563f None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:30.327198 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7351595f-d112-431e-989c-a7039655563f None admin] PATH_INFO: `/v3/projects/af04daa468b64160a6176dcab29502a7/users/f9116f47889f4db39e0203d6cdae1593/roles/c4bf019fc6c44c71a6c984f6d0bab75f` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:30.333215 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7351595f-d112-431e-989c-a7039655563f None admin] RBAC: Authorizing `identity:create_grant(project_id=af04daa468b64160a6176dcab29502a7, user_id=f9116f47889f4db39e0203d6cdae1593, role_id=c4bf019fc6c44c71a6c984f6d0bab75f)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:30.336130 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7351595f-d112-431e-989c-a7039655563f None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:30.363131 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 289/577] 10.222.0.18 () {58 vars in 1439 bytes} [Wed Nov 2 07:17:30 2022] PUT /identity/v3/projects/af04daa468b64160a6176dcab29502a7/users/f9116f47889f4db39e0203d6cdae1593/roles/c4bf019fc6c44c71a6c984f6d0bab75f => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:17:30.465033 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80f8857-d442-413b-a04d-08e00c8e972a None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:30.468770 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:30.488317 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f70e3c1-1c58-4fdd-bf17-cf9fad88a89a None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:30.488646 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f70e3c1-1c58-4fdd-bf17-cf9fad88a89a None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:30.488906 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f70e3c1-1c58-4fdd-bf17-cf9fad88a89a None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:30.490719 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 290/578] 10.222.0.18 () {54 vars in 927 bytes} [Wed Nov 2 07:17:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Nov 02 07:17:30.496203 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc1ddc0-07f3-47d1-864b-09fd10d03856 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:30.496512 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc1ddc0-07f3-47d1-864b-09fd10d03856 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:30.496762 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc1ddc0-07f3-47d1-864b-09fd10d03856 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:30.513603 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072b46bc-9c8a-4f99-94a5-7408a3d920b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:30.514341 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072b46bc-9c8a-4f99-94a5-7408a3d920b4 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:30.514643 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072b46bc-9c8a-4f99-94a5-7408a3d920b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:30.514920 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072b46bc-9c8a-4f99-94a5-7408a3d920b4 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:30.515606 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072b46bc-9c8a-4f99-94a5-7408a3d920b4 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:30.516275 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-3bc1ddc0-07f3-47d1-864b-09fd10d03856 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:17:30.516977 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072b46bc-9c8a-4f99-94a5-7408a3d920b4 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:30.517464 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-3bc1ddc0-07f3-47d1-864b-09fd10d03856 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:17:30.532396 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-072b46bc-9c8a-4f99-94a5-7408a3d920b4 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Nov 02 07:17:30.532396 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:17:30.532396 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Nov 02 07:17:30.532396 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Nov 02 07:17:30.532396 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Nov 02 07:17:30.532396 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Nov 02 07:17:30.532396 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Nov 02 07:17:30.532396 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:17:30.532396 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Nov 02 07:17:30.532396 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Nov 02 07:17:30.532396 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 07:17:30.532396 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Nov 02 07:17:30.532396 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:17:30.532396 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:17:30.532396 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:17:30.532396 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:17:30.532396 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:17:30.533420 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:17:30.533420 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:17:30.533420 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:17:30.533420 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:17:30.533420 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:17:30.533420 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:17:30.533420 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Nov 02 07:17:30.533420 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_service(service_id) Nov 02 07:17:30.533420 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Nov 02 07:17:30.533420 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Nov 02 07:17:30.533420 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:17:30.533420 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:17:30.533420 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:17:30.533420 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:17:30.533420 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:17:30.533420 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:17:30.534295 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:17:30.534295 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Nov 02 07:17:30.534295 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:17:30.534295 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:17:30.534295 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:17:30.534295 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:17:30.534295 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:17:30.534295 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:17:30.534295 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:17:30.534295 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:17:30.534295 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Nov 02 07:17:30.534295 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:17:30.534295 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Nov 02 07:17:30.534295 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Nov 02 07:17:30.535331 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:30.535405 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 289/579] 10.222.0.18 () {56 vars in 1129 bytes} [Wed Nov 2 07:17:30 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 07:17:30.542458 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072b46bc-9c8a-4f99-94a5-7408a3d920b4 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:30.637466 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 291/580] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Nov 2 07:17:30 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Nov 02 07:17:30.703346 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072b46bc-9c8a-4f99-94a5-7408a3d920b4 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:30.742429 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e47d4ea-e547-4ea7-8811-56a995b3d121 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:30.742946 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e47d4ea-e547-4ea7-8811-56a995b3d121 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:30.743227 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e47d4ea-e547-4ea7-8811-56a995b3d121 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:30.743493 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e47d4ea-e547-4ea7-8811-56a995b3d121 None admin] PATH_INFO: `/v3/services` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:30.744116 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e47d4ea-e547-4ea7-8811-56a995b3d121 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:17:30.744409 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e47d4ea-e547-4ea7-8811-56a995b3d121 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:30.745836 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e47d4ea-e547-4ea7-8811-56a995b3d121 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:30.750987 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 290/581] 10.222.0.18 () {56 vars in 1124 bytes} [Wed Nov 2 07:17:30 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:17:30.756711 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc1ddc0-07f3-47d1-864b-09fd10d03856 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:30.874286 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc1ddc0-07f3-47d1-864b-09fd10d03856 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:30.923074 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a30d837-4218-4ffe-b8af-b31367a9ac48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:30.923719 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e47d4ea-e547-4ea7-8811-56a995b3d121 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:30.923806 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a30d837-4218-4ffe-b8af-b31367a9ac48 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:30.924084 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a30d837-4218-4ffe-b8af-b31367a9ac48 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:30.924243 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a30d837-4218-4ffe-b8af-b31367a9ac48 None admin] PATH_INFO: `/v3/services` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:30.924826 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a30d837-4218-4ffe-b8af-b31367a9ac48 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:17:30.925120 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a30d837-4218-4ffe-b8af-b31367a9ac48 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:30.926524 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a30d837-4218-4ffe-b8af-b31367a9ac48 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:30.933318 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 292/582] 10.222.0.18 () {56 vars in 1124 bytes} [Wed Nov 2 07:17:30 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:17:30.938835 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a30d837-4218-4ffe-b8af-b31367a9ac48 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:30.958869 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:30.988373 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:31.069044 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e47d4ea-e547-4ea7-8811-56a995b3d121 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:31.075807 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a30d837-4218-4ffe-b8af-b31367a9ac48 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:31.104400 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d187642-7a29-463e-9b16-18fa198386f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:31.105186 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d187642-7a29-463e-9b16-18fa198386f8 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:31.105465 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d187642-7a29-463e-9b16-18fa198386f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:31.105732 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d187642-7a29-463e-9b16-18fa198386f8 None admin] PATH_INFO: `/v3/services/network` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:31.106354 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d187642-7a29-463e-9b16-18fa198386f8 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:31.107944 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d187642-7a29-463e-9b16-18fa198386f8 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:31.112900 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58fc612-1b38-4654-a74d-3ea77d26679b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:31.113510 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58fc612-1b38-4654-a74d-3ea77d26679b None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:31.113789 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58fc612-1b38-4654-a74d-3ea77d26679b None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:31.114122 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58fc612-1b38-4654-a74d-3ea77d26679b None admin] PATH_INFO: `/v3/endpoints` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:31.114924 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58fc612-1b38-4654-a74d-3ea77d26679b None admin] RBAC: Adding query filter params (interface=public, service_id=991f4b3b3fe3481e90a9de98495e0c55, region_id=RegionOne) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:17:31.115224 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58fc612-1b38-4654-a74d-3ea77d26679b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:31.116584 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58fc612-1b38-4654-a74d-3ea77d26679b None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:31.116966 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-4d187642-7a29-463e-9b16-18fa198386f8 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Nov 02 07:17:31.116966 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:17:31.116966 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Nov 02 07:17:31.116966 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Nov 02 07:17:31.116966 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Nov 02 07:17:31.116966 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Nov 02 07:17:31.116966 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Nov 02 07:17:31.116966 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:17:31.116966 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Nov 02 07:17:31.116966 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Nov 02 07:17:31.116966 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 07:17:31.116966 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Nov 02 07:17:31.116966 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:17:31.116966 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:17:31.116966 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:17:31.116966 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:17:31.116966 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:17:31.117939 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:17:31.117939 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:17:31.117939 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:17:31.117939 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:17:31.117939 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:17:31.117939 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:17:31.117939 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Nov 02 07:17:31.117939 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_service(service_id) Nov 02 07:17:31.117939 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Nov 02 07:17:31.117939 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Nov 02 07:17:31.117939 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:17:31.117939 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:17:31.117939 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:17:31.117939 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:17:31.117939 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:17:31.117939 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:17:31.118835 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:17:31.118835 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Nov 02 07:17:31.118835 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:17:31.118835 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:17:31.118835 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:17:31.118835 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:17:31.118835 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:17:31.118835 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:17:31.118835 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:17:31.118835 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:17:31.118835 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Nov 02 07:17:31.118835 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:17:31.118835 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Nov 02 07:17:31.118835 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Nov 02 07:17:31.118835 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 291/583] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Nov 2 07:17:30 2022] GET /identity/v3/services/network => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 07:17:31.120118 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166c3cbb-f9a1-40ce-b528-5de3c0edcc4b None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:31.120118 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166c3cbb-f9a1-40ce-b528-5de3c0edcc4b None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:31.120403 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166c3cbb-f9a1-40ce-b528-5de3c0edcc4b None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:31.121879 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 292/584] 10.222.0.18 () {54 vars in 927 bytes} [Wed Nov 2 07:17:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:17:31.124859 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 293/585] 10.222.0.18 () {56 vars in 1251 bytes} [Wed Nov 2 07:17:30 2022] GET /identity/v3/endpoints?service_id=991f4b3b3fe3481e90a9de98495e0c55&interface=public®ion_id=RegionOne => generated 194 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:17:31.126663 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166c3cbb-f9a1-40ce-b528-5de3c0edcc4b None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:31.130762 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58fc612-1b38-4654-a74d-3ea77d26679b None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:31.216175 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:31.241124 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58fc612-1b38-4654-a74d-3ea77d26679b None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:31.251203 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:31.259643 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166c3cbb-f9a1-40ce-b528-5de3c0edcc4b None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:31.289415 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc84736-4b72-423b-ba6a-aa88ed560004 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:31.290680 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc84736-4b72-423b-ba6a-aa88ed560004 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:31.291411 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc84736-4b72-423b-ba6a-aa88ed560004 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:31.291929 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc84736-4b72-423b-ba6a-aa88ed560004 None admin] PATH_INFO: `/v3/services` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:31.293316 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc84736-4b72-423b-ba6a-aa88ed560004 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:31.295254 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc84736-4b72-423b-ba6a-aa88ed560004 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:31.296938 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c18ae4-0f03-4ba7-9fa1-6f09de3b8371 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:31.297721 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c18ae4-0f03-4ba7-9fa1-6f09de3b8371 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:31.298004 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c18ae4-0f03-4ba7-9fa1-6f09de3b8371 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:31.298402 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c18ae4-0f03-4ba7-9fa1-6f09de3b8371 None admin] PATH_INFO: `/v3/services` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:31.299092 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c18ae4-0f03-4ba7-9fa1-6f09de3b8371 None admin] RBAC: Adding query filter params (name=network) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:17:31.299466 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c18ae4-0f03-4ba7-9fa1-6f09de3b8371 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:31.300861 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c18ae4-0f03-4ba7-9fa1-6f09de3b8371 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:31.303900 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 294/586] 10.222.0.18 () {56 vars in 1087 bytes} [Wed Nov 2 07:17:31 2022] GET /identity/v3/services => generated 772 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:17:31.305304 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740db57b-e2a0-42e3-be5b-b6a9e687e52d None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:31.305763 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740db57b-e2a0-42e3-be5b-b6a9e687e52d None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:31.306194 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740db57b-e2a0-42e3-be5b-b6a9e687e52d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:31.314716 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 293/587] 10.222.0.18 () {56 vars in 1112 bytes} [Wed Nov 2 07:17:31 2022] GET /identity/v3/services?name=network => generated 124 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:17:31.320554 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c18ae4-0f03-4ba7-9fa1-6f09de3b8371 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:31.328766 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-740db57b-e2a0-42e3-be5b-b6a9e687e52d None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:17:31.330246 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-740db57b-e2a0-42e3-be5b-b6a9e687e52d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:17:31.423169 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 295/588] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Nov 2 07:17:31 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 07:17:31.518897 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c18ae4-0f03-4ba7-9fa1-6f09de3b8371 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:31.567615 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533bd810-ec04-45ad-945e-36e0c66f53f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:31.568598 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533bd810-ec04-45ad-945e-36e0c66f53f0 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:31.569097 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533bd810-ec04-45ad-945e-36e0c66f53f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:31.569574 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533bd810-ec04-45ad-945e-36e0c66f53f0 None admin] PATH_INFO: `/v3/services` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:31.570465 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533bd810-ec04-45ad-945e-36e0c66f53f0 None admin] RBAC: Adding query filter params (type=network) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:17:31.571047 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533bd810-ec04-45ad-945e-36e0c66f53f0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:31.572656 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533bd810-ec04-45ad-945e-36e0c66f53f0 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:31.580089 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 294/589] 10.222.0.18 () {56 vars in 1112 bytes} [Wed Nov 2 07:17:31 2022] GET /identity/v3/services?type=network => generated 350 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:17:31.586690 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740db57b-e2a0-42e3-be5b-b6a9e687e52d None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:31.660500 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:31.692245 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740db57b-e2a0-42e3-be5b-b6a9e687e52d None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:31.726677 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce1757a-a7e9-40cd-8d8d-984e8d20082e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:31.727281 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce1757a-a7e9-40cd-8d8d-984e8d20082e None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:31.727559 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce1757a-a7e9-40cd-8d8d-984e8d20082e None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:31.728083 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce1757a-a7e9-40cd-8d8d-984e8d20082e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:31.728774 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce1757a-a7e9-40cd-8d8d-984e8d20082e None admin] RBAC: Adding query filter params (interface=public, service_id=579b130fd62248a293c7d4d848cbb54c, region_id=RegionOne) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:17:31.729071 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce1757a-a7e9-40cd-8d8d-984e8d20082e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:31.730437 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce1757a-a7e9-40cd-8d8d-984e8d20082e None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:31.736072 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 296/590] 10.222.0.18 () {56 vars in 1251 bytes} [Wed Nov 2 07:17:31 2022] GET /identity/v3/endpoints?service_id=579b130fd62248a293c7d4d848cbb54c&interface=public®ion_id=RegionOne => generated 194 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:17:31.742664 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533bd810-ec04-45ad-945e-36e0c66f53f0 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:31.776590 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:31.782291 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce1757a-a7e9-40cd-8d8d-984e8d20082e None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:31.898745 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce1757a-a7e9-40cd-8d8d-984e8d20082e None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:31.935556 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ef83da-dc7a-42b0-b248-93ae790205d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:31.936214 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ef83da-dc7a-42b0-b248-93ae790205d0 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:31.936462 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ef83da-dc7a-42b0-b248-93ae790205d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:31.936716 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ef83da-dc7a-42b0-b248-93ae790205d0 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:31.937299 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ef83da-dc7a-42b0-b248-93ae790205d0 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:31.938679 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ef83da-dc7a-42b0-b248-93ae790205d0 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:31.951031 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-e9ef83da-dc7a-42b0-b248-93ae790205d0 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Nov 02 07:17:31.951031 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:17:31.951031 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Nov 02 07:17:31.951031 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Nov 02 07:17:31.951031 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Nov 02 07:17:31.951031 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Nov 02 07:17:31.951031 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Nov 02 07:17:31.951031 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:17:31.951031 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Nov 02 07:17:31.951031 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:17:31.951031 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 07:17:31.951031 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:17:31.951031 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:17:31.951031 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:17:31.951031 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:17:31.951031 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:17:31.952067 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:17:31.952067 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:17:31.952067 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:17:31.952067 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:17:31.952067 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:17:31.952067 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:17:31.952067 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:17:31.952067 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Nov 02 07:17:31.952067 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_service(service_id) Nov 02 07:17:31.952067 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Nov 02 07:17:31.952067 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Nov 02 07:17:31.952067 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:17:31.952067 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:17:31.952067 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:17:31.952067 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:17:31.952067 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:17:31.952067 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:17:31.953018 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:17:31.953018 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:17:31.953018 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:17:31.953018 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:17:31.953018 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:17:31.953018 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:17:31.953018 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:17:31.953018 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:17:31.953018 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:17:31.953018 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:17:31.953018 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Nov 02 07:17:31.953018 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:17:31.953018 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Nov 02 07:17:31.953018 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:17:31.953018 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 297/591] 10.222.0.18 () {56 vars in 1132 bytes} [Wed Nov 2 07:17:31 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 07:17:31.954255 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:31.959303 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ef83da-dc7a-42b0-b248-93ae790205d0 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:31.993984 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533bd810-ec04-45ad-945e-36e0c66f53f0 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:32.072568 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ef83da-dc7a-42b0-b248-93ae790205d0 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:32.107447 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a2d9d5-4543-4b8b-b9cf-63511e27350b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:32.108119 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a2d9d5-4543-4b8b-b9cf-63511e27350b None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:32.108397 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a2d9d5-4543-4b8b-b9cf-63511e27350b None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:32.108661 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a2d9d5-4543-4b8b-b9cf-63511e27350b None admin] PATH_INFO: `/v3/services` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:32.109260 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a2d9d5-4543-4b8b-b9cf-63511e27350b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:32.110387 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4ab5fe-1805-4e0b-b83b-3330e0984a07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:32.110637 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a2d9d5-4543-4b8b-b9cf-63511e27350b None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:32.110995 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4ab5fe-1805-4e0b-b83b-3330e0984a07 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:32.111273 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4ab5fe-1805-4e0b-b83b-3330e0984a07 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:32.111543 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4ab5fe-1805-4e0b-b83b-3330e0984a07 None admin] PATH_INFO: `/v3/services` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:32.112340 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4ab5fe-1805-4e0b-b83b-3330e0984a07 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:17:32.112642 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4ab5fe-1805-4e0b-b83b-3330e0984a07 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:32.114742 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4ab5fe-1805-4e0b-b83b-3330e0984a07 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:32.126263 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 298/592] 10.222.0.18 () {56 vars in 1126 bytes} [Wed Nov 2 07:17:31 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:17:32.127503 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4ab5fe-1805-4e0b-b83b-3330e0984a07 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:32.129030 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 295/593] 10.222.0.18 () {56 vars in 1087 bytes} [Wed Nov 2 07:17:31 2022] GET /identity/v3/services => generated 772 bytes in 387 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:17:32.136949 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493e2010-58bc-4518-91a7-1cbc244495a2 None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:32.137294 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493e2010-58bc-4518-91a7-1cbc244495a2 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:32.137504 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493e2010-58bc-4518-91a7-1cbc244495a2 None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:32.139243 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 296/594] 10.222.0.18 () {54 vars in 927 bytes} [Wed Nov 2 07:17:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:17:32.151986 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f538e051-1b75-4dd7-b609-8f3e26466f33 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:32.152190 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f538e051-1b75-4dd7-b609-8f3e26466f33 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:32.152428 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f538e051-1b75-4dd7-b609-8f3e26466f33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:32.195703 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-f538e051-1b75-4dd7-b609-8f3e26466f33 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:17:32.196927 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-f538e051-1b75-4dd7-b609-8f3e26466f33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:17:32.219706 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:32.286474 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4ab5fe-1805-4e0b-b83b-3330e0984a07 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:32.339953 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 297/595] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Nov 2 07:17:32 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 07:17:32.350297 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7289113-b596-4671-9b4e-afbe47e60f1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:32.350975 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7289113-b596-4671-9b4e-afbe47e60f1a None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:32.351249 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7289113-b596-4671-9b4e-afbe47e60f1a None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:32.351516 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7289113-b596-4671-9b4e-afbe47e60f1a None admin] PATH_INFO: `/v3/services` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:32.352140 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7289113-b596-4671-9b4e-afbe47e60f1a None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:17:32.352431 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7289113-b596-4671-9b4e-afbe47e60f1a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:32.354044 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7289113-b596-4671-9b4e-afbe47e60f1a None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:32.362482 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 299/596] 10.222.0.18 () {56 vars in 1126 bytes} [Wed Nov 2 07:17:32 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:17:32.653808 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f538e051-1b75-4dd7-b609-8f3e26466f33 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:32.676439 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:32.782833 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f538e051-1b75-4dd7-b609-8f3e26466f33 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:32.820771 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa4dd44-52a9-469e-b352-fe5e6ee4cadf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:32.821407 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa4dd44-52a9-469e-b352-fe5e6ee4cadf None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:32.821686 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa4dd44-52a9-469e-b352-fe5e6ee4cadf None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:32.822095 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa4dd44-52a9-469e-b352-fe5e6ee4cadf None admin] PATH_INFO: `/v3/roles/service` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:32.833509 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa4dd44-52a9-469e-b352-fe5e6ee4cadf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:17:32.838485 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-afa4dd44-52a9-469e-b352-fe5e6ee4cadf None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Nov 02 07:17:32.838485 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:17:32.838485 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:17:32.838485 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:17:32.838485 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:17:32.838485 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:17:32.838485 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:17:32.838485 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:17:32.838485 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:17:32.838485 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:17:32.838485 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:17:32.838485 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:17:32.838485 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:17:32.838485 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:17:32.838485 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:17:32.838485 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:17:32.839417 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:17:32.839417 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:17:32.839417 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:17:32.839417 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:17:32.839417 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:17:32.839417 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:17:32.839417 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:17:32.839417 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:17:32.839417 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:17:32.839417 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:17:32.839417 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:17:32.839417 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Nov 02 07:17:32.839417 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:17:32.839417 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:17:32.839417 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:17:32.839417 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:17:32.840259 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:17:32.840259 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:17:32.840259 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:17:32.840259 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:17:32.840259 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:17:32.840259 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:17:32.840259 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:17:32.840259 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:17:32.840259 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:17:32.840259 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:17:32.840259 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Nov 02 07:17:32.840259 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Nov 02 07:17:32.840259 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 298/597] 10.222.0.18 () {56 vars in 1102 bytes} [Wed Nov 2 07:17:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:17:32.846920 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7289113-b596-4671-9b4e-afbe47e60f1a None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:32.983836 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7289113-b596-4671-9b4e-afbe47e60f1a None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:33.025193 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:33.033746 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5396998-a357-4f44-8ae7-77c1df264b67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:33.034824 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5396998-a357-4f44-8ae7-77c1df264b67 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:33.035544 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5396998-a357-4f44-8ae7-77c1df264b67 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:33.035920 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5396998-a357-4f44-8ae7-77c1df264b67 None admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:33.036625 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5396998-a357-4f44-8ae7-77c1df264b67 None admin] RBAC: Adding query filter params (name=service) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:17:33.036943 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5396998-a357-4f44-8ae7-77c1df264b67 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:33.038466 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5396998-a357-4f44-8ae7-77c1df264b67 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:33.051084 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 300/598] 10.222.0.18 () {56 vars in 1103 bytes} [Wed Nov 2 07:17:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:17:33.057675 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa4dd44-52a9-469e-b352-fe5e6ee4cadf None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:33.166047 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c317ed16-77e7-40db-b859-674d6e9eee2c None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:33.166557 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c317ed16-77e7-40db-b859-674d6e9eee2c None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:33.166947 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c317ed16-77e7-40db-b859-674d6e9eee2c None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:33.168610 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 301/599] 10.222.0.18 () {54 vars in 927 bytes} [Wed Nov 2 07:17:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Nov 02 07:17:33.173599 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bef5b7-a668-49f0-83e5-91122800b920 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:33.173977 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bef5b7-a668-49f0-83e5-91122800b920 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:33.174342 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bef5b7-a668-49f0-83e5-91122800b920 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:33.175433 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:33.203812 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa4dd44-52a9-469e-b352-fe5e6ee4cadf None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:33.215170 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-28bef5b7-a668-49f0-83e5-91122800b920 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:17:33.216572 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-28bef5b7-a668-49f0-83e5-91122800b920 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:17:33.268212 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556eebe4-d4ae-4958-b984-4a8b40c97d33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:33.272316 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556eebe4-d4ae-4958-b984-4a8b40c97d33 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:33.272753 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556eebe4-d4ae-4958-b984-4a8b40c97d33 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:33.274075 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556eebe4-d4ae-4958-b984-4a8b40c97d33 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:33.299285 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556eebe4-d4ae-4958-b984-4a8b40c97d33 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:33.301043 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556eebe4-d4ae-4958-b984-4a8b40c97d33 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:33.328173 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-556eebe4-d4ae-4958-b984-4a8b40c97d33 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:17:33.328173 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:17:33.328173 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:17:33.328173 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:17:33.328173 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:17:33.328173 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:17:33.328173 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:17:33.328173 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:17:33.328173 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:17:33.328173 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:17:33.328173 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:17:33.328173 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:17:33.328173 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:17:33.328173 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:17:33.328173 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:17:33.328173 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:17:33.329449 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:17:33.329449 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:17:33.329449 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:17:33.329449 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:17:33.329449 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:17:33.329449 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:17:33.329449 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:17:33.329449 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Nov 02 07:17:33.329449 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:17:33.329449 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:17:33.329449 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:17:33.329449 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:17:33.329449 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:17:33.329449 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:17:33.329449 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:17:33.329449 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:17:33.329449 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:17:33.330570 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:17:33.330570 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:17:33.330570 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Nov 02 07:17:33.339292 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 299/600] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Nov 2 07:17:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:17:33.343693 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556eebe4-d4ae-4958-b984-4a8b40c97d33 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:33.368106 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 302/601] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Nov 2 07:17:33 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Nov 02 07:17:33.458018 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556eebe4-d4ae-4958-b984-4a8b40c97d33 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:33.493085 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e310a8-2553-47f0-b685-f58fb9ca327e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:33.493957 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e310a8-2553-47f0-b685-f58fb9ca327e None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:33.494425 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e310a8-2553-47f0-b685-f58fb9ca327e None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:33.494872 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:33.495323 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e310a8-2553-47f0-b685-f58fb9ca327e None admin] PATH_INFO: `/v3/domains` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:33.496111 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e310a8-2553-47f0-b685-f58fb9ca327e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:17:33.496616 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e310a8-2553-47f0-b685-f58fb9ca327e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:33.498238 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e310a8-2553-47f0-b685-f58fb9ca327e None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:33.511912 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 300/602] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Nov 2 07:17:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:17:33.517797 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bef5b7-a668-49f0-83e5-91122800b920 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:33.680523 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bef5b7-a668-49f0-83e5-91122800b920 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:33.709819 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:33.730822 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7183bdcd-e42c-4b95-b051-409c734f5a37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:33.731794 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7183bdcd-e42c-4b95-b051-409c734f5a37 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:33.732274 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7183bdcd-e42c-4b95-b051-409c734f5a37 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:33.732740 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7183bdcd-e42c-4b95-b051-409c734f5a37 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:33.745913 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7183bdcd-e42c-4b95-b051-409c734f5a37 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:33.747655 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7183bdcd-e42c-4b95-b051-409c734f5a37 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:33.766050 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-7183bdcd-e42c-4b95-b051-409c734f5a37 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Nov 02 07:17:33.766050 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:17:33.766050 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:17:33.766050 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:17:33.766050 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:17:33.766050 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:17:33.766050 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:17:33.766050 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:17:33.766050 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:17:33.766050 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:17:33.766050 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:17:33.766050 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:17:33.766050 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Nov 02 07:17:33.766050 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_user(user_id) Nov 02 07:17:33.766050 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Nov 02 07:17:33.766050 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Nov 02 07:17:33.767077 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:17:33.767077 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:17:33.767077 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Nov 02 07:17:33.767077 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:17:33.767077 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Nov 02 07:17:33.767077 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:17:33.767077 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:17:33.767077 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:17:33.767077 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:17:33.767077 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:17:33.767077 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:17:33.767077 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:17:33.767077 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:17:33.767077 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:17:33.767077 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:17:33.767077 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:17:33.767077 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:17:33.767077 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:17:33.768000 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:17:33.768000 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:17:33.768000 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Nov 02 07:17:33.768000 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Nov 02 07:17:33.768000 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Nov 02 07:17:33.768000 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Nov 02 07:17:33.768000 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Nov 02 07:17:33.768000 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Nov 02 07:17:33.768000 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Nov 02 07:17:33.768000 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:17:33.771217 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 303/603] 10.222.0.18 () {56 vars in 1117 bytes} [Wed Nov 2 07:17:33 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:17:33.776978 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e310a8-2553-47f0-b685-f58fb9ca327e None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:33.875032 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7183bdcd-e42c-4b95-b051-409c734f5a37 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:33.961149 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e310a8-2553-47f0-b685-f58fb9ca327e None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:33.997163 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7183bdcd-e42c-4b95-b051-409c734f5a37 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:34.033063 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac804da2-61b2-4297-b320-7f6703e28354 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:34.033691 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac804da2-61b2-4297-b320-7f6703e28354 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:34.034079 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac804da2-61b2-4297-b320-7f6703e28354 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:34.034449 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac804da2-61b2-4297-b320-7f6703e28354 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:34.035028 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ea9326-8442-4f8e-bb73-0c601a48a680 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:34.035829 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ea9326-8442-4f8e-bb73-0c601a48a680 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:34.036617 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ea9326-8442-4f8e-bb73-0c601a48a680 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:34.036900 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ea9326-8442-4f8e-bb73-0c601a48a680 None admin] PATH_INFO: `/v3/users` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:34.037637 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ea9326-8442-4f8e-bb73-0c601a48a680 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:17:34.037945 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ea9326-8442-4f8e-bb73-0c601a48a680 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:34.039399 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ea9326-8442-4f8e-bb73-0c601a48a680 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:34.045220 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac804da2-61b2-4297-b320-7f6703e28354 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:17:34.050964 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-ac804da2-61b2-4297-b320-7f6703e28354 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 07:17:34.050964 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:17:34.050964 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:17:34.050964 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:17:34.050964 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:17:34.050964 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:17:34.050964 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:17:34.050964 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:17:34.050964 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:17:34.050964 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:17:34.050964 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:17:34.050964 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:17:34.050964 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:17:34.050964 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:17:34.050964 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:17:34.050964 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:17:34.052644 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:17:34.052644 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:17:34.052644 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:17:34.052644 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:17:34.052644 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:17:34.052644 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:17:34.052644 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:17:34.052644 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:17:34.052644 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:17:34.052644 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:17:34.052644 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:17:34.052644 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:17:34.052644 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:17:34.052644 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:17:34.052644 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:17:34.052644 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:17:34.053487 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:17:34.053487 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:17:34.053487 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:17:34.053487 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:17:34.053487 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:17:34.053487 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:17:34.053487 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:17:34.053487 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:17:34.053487 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:17:34.053487 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:17:34.053487 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 07:17:34.053487 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:17:34.053487 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 304/604] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Nov 2 07:17:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:17:34.061012 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac804da2-61b2-4297-b320-7f6703e28354 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:34.094287 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 301/605] 10.222.0.18 () {56 vars in 1149 bytes} [Wed Nov 2 07:17:33 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:17:34.101017 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ea9326-8442-4f8e-bb73-0c601a48a680 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:34.118471 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:34.206612 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac804da2-61b2-4297-b320-7f6703e28354 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:34.210375 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:34.247037 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6f0729-7d9d-447a-8eca-3d61f2c41fae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:34.247664 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6f0729-7d9d-447a-8eca-3d61f2c41fae None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:34.247949 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6f0729-7d9d-447a-8eca-3d61f2c41fae None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:34.248384 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6f0729-7d9d-447a-8eca-3d61f2c41fae None admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:34.248993 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6f0729-7d9d-447a-8eca-3d61f2c41fae None admin] RBAC: Adding query filter params (name=admin) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:17:34.249291 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6f0729-7d9d-447a-8eca-3d61f2c41fae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:34.250694 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6f0729-7d9d-447a-8eca-3d61f2c41fae None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:34.261653 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 305/606] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Nov 2 07:17:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:17:34.263980 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0460c689-c125-4d79-a096-ac57d96eb835 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:34.263980 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0460c689-c125-4d79-a096-ac57d96eb835 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:34.263980 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0460c689-c125-4d79-a096-ac57d96eb835 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:34.264768 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 306/607] 10.222.0.18 () {54 vars in 927 bytes} [Wed Nov 2 07:17:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:17:34.269504 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0460c689-c125-4d79-a096-ac57d96eb835 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:34.295246 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ea9326-8442-4f8e-bb73-0c601a48a680 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:34.338866 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7ad773-5a54-4cb1-a679-e448134de11a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:34.340448 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7ad773-5a54-4cb1-a679-e448134de11a None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:34.340945 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7ad773-5a54-4cb1-a679-e448134de11a None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:34.341390 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7ad773-5a54-4cb1-a679-e448134de11a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:34.394111 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0460c689-c125-4d79-a096-ac57d96eb835 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:34.396677 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:34.423997 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7ad773-5a54-4cb1-a679-e448134de11a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:34.425650 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7ad773-5a54-4cb1-a679-e448134de11a None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:34.431901 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00d3143-ebc7-407f-af43-92802bdc962d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:34.432838 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00d3143-ebc7-407f-af43-92802bdc962d None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:34.433288 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00d3143-ebc7-407f-af43-92802bdc962d None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:34.433752 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00d3143-ebc7-407f-af43-92802bdc962d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:34.445425 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00d3143-ebc7-407f-af43-92802bdc962d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:34.447165 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00d3143-ebc7-407f-af43-92802bdc962d None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:34.462451 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-f00d3143-ebc7-407f-af43-92802bdc962d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:17:34.462451 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:17:34.462451 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:17:34.462451 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:17:34.462451 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:17:34.462451 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:17:34.462451 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:17:34.462451 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:17:34.462451 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:17:34.462451 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:17:34.462451 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:17:34.462451 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:17:34.462451 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:17:34.462451 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:17:34.462451 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:17:34.462451 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:17:34.463496 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:17:34.463496 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:17:34.463496 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:17:34.463496 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:17:34.463496 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:17:34.463496 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:17:34.463496 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:17:34.463496 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:17:34.463496 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:17:34.463496 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:17:34.463496 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:17:34.463496 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:17:34.463496 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:17:34.463496 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:17:34.463496 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:17:34.463496 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:17:34.463496 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:17:34.464403 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:17:34.464403 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:17:34.464403 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:17:34.470973 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 307/608] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Nov 2 07:17:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:17:34.472446 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c868baf-0b3e-4c21-92bf-de0b6ae25503 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:34.472917 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c868baf-0b3e-4c21-92bf-de0b6ae25503 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:34.473513 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c868baf-0b3e-4c21-92bf-de0b6ae25503 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:34.479904 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:34.492980 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-9c868baf-0b3e-4c21-92bf-de0b6ae25503 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:17:34.494478 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-9c868baf-0b3e-4c21-92bf-de0b6ae25503 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:17:34.544056 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 302/609] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Nov 2 07:17:34 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Nov 02 07:17:34.545139 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7ad773-5a54-4cb1-a679-e448134de11a None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:34.564529 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 308/610] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Nov 2 07:17:34 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 07:17:34.565629 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c868baf-0b3e-4c21-92bf-de0b6ae25503 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:34.662074 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:34.679256 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c868baf-0b3e-4c21-92bf-de0b6ae25503 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:34.728417 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7ad773-5a54-4cb1-a679-e448134de11a None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:34.747468 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c10c0a5-4100-4259-bf56-742c45383bf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:34.748408 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c10c0a5-4100-4259-bf56-742c45383bf6 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:34.748961 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c10c0a5-4100-4259-bf56-742c45383bf6 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:34.749442 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c10c0a5-4100-4259-bf56-742c45383bf6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:34.768624 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c10c0a5-4100-4259-bf56-742c45383bf6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:34.771880 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c10c0a5-4100-4259-bf56-742c45383bf6 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:34.790575 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-9c10c0a5-4100-4259-bf56-742c45383bf6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:17:34.790575 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:17:34.790575 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:17:34.790575 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:17:34.790575 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:17:34.790575 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:17:34.790575 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:17:34.790575 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:17:34.790575 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:17:34.790575 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:17:34.790575 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:17:34.790575 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:17:34.790575 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:17:34.790575 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:17:34.790575 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:17:34.790575 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:17:34.791580 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:17:34.791580 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:17:34.791580 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:17:34.791580 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:17:34.791580 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:17:34.791580 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:17:34.791580 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:17:34.791580 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:17:34.791580 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:17:34.791580 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:17:34.791580 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:17:34.791580 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:17:34.791580 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:17:34.791580 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:17:34.791580 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:17:34.791580 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:17:34.791580 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:17:34.792463 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:17:34.792463 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:17:34.792463 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:17:34.794469 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e4d6b1-32be-47b7-82c7-1ea29f57c9b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:34.794950 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 309/611] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Nov 2 07:17:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:17:34.795175 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e4d6b1-32be-47b7-82c7-1ea29f57c9b6 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:34.795521 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e4d6b1-32be-47b7-82c7-1ea29f57c9b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:34.795797 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e4d6b1-32be-47b7-82c7-1ea29f57c9b6 None admin] PATH_INFO: `/v3/domains` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:34.796440 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e4d6b1-32be-47b7-82c7-1ea29f57c9b6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:17:34.796741 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e4d6b1-32be-47b7-82c7-1ea29f57c9b6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:34.798275 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e4d6b1-32be-47b7-82c7-1ea29f57c9b6 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:34.808753 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1916339f-3cac-4fec-8c7b-a899ca091c77 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:34.809332 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1916339f-3cac-4fec-8c7b-a899ca091c77 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:34.809863 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1916339f-3cac-4fec-8c7b-a899ca091c77 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:34.811630 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 310/612] 10.222.0.18 () {54 vars in 927 bytes} [Wed Nov 2 07:17:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:17:34.812531 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1916339f-3cac-4fec-8c7b-a899ca091c77 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:34.818235 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 303/613] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Nov 2 07:17:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:17:34.818235 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5778ee7-221b-428c-bf2b-1cfec64e25c0 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:34.818235 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5778ee7-221b-428c-bf2b-1cfec64e25c0 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:34.818235 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5778ee7-221b-428c-bf2b-1cfec64e25c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:34.837880 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-a5778ee7-221b-428c-bf2b-1cfec64e25c0 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:17:34.839189 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-a5778ee7-221b-428c-bf2b-1cfec64e25c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:17:34.841179 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:34.920261 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 304/614] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Nov 2 07:17:34 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 07:17:34.921248 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5778ee7-221b-428c-bf2b-1cfec64e25c0 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:34.931967 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1916339f-3cac-4fec-8c7b-a899ca091c77 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:34.977060 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e1c1b9-4f37-470b-8319-95b99e269230 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:34.977934 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e1c1b9-4f37-470b-8319-95b99e269230 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:34.978208 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e1c1b9-4f37-470b-8319-95b99e269230 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:34.978509 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e1c1b9-4f37-470b-8319-95b99e269230 None admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:34.979156 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e1c1b9-4f37-470b-8319-95b99e269230 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:17:34.979452 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e1c1b9-4f37-470b-8319-95b99e269230 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:34.980827 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e1c1b9-4f37-470b-8319-95b99e269230 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:34.983472 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:34.995225 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 311/615] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Nov 2 07:17:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:17:34.996642 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e1c1b9-4f37-470b-8319-95b99e269230 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:35.049727 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5778ee7-221b-428c-bf2b-1cfec64e25c0 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:35.088258 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3a0b03-4205-416c-ab9d-f831fac9e0a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:35.088903 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3a0b03-4205-416c-ab9d-f831fac9e0a1 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:35.089183 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3a0b03-4205-416c-ab9d-f831fac9e0a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:35.089450 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3a0b03-4205-416c-ab9d-f831fac9e0a1 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:35.096646 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3a0b03-4205-416c-ab9d-f831fac9e0a1 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:35.098104 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3a0b03-4205-416c-ab9d-f831fac9e0a1 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:35.099119 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e1c1b9-4f37-470b-8319-95b99e269230 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:35.100147 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:35.112554 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-7a3a0b03-4205-416c-ab9d-f831fac9e0a1 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Nov 02 07:17:35.112554 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:17:35.112554 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:17:35.112554 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:17:35.112554 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:17:35.112554 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:17:35.112554 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:17:35.112554 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:17:35.112554 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:17:35.112554 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:17:35.112554 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:17:35.112554 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:17:35.112554 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Nov 02 07:17:35.112554 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_user(user_id) Nov 02 07:17:35.112554 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Nov 02 07:17:35.112554 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Nov 02 07:17:35.113514 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:17:35.113514 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:17:35.113514 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Nov 02 07:17:35.113514 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:17:35.113514 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Nov 02 07:17:35.113514 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:17:35.113514 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:17:35.113514 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:17:35.113514 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:17:35.113514 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:17:35.113514 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:17:35.113514 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Nov 02 07:17:35.113514 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:17:35.113514 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:17:35.113514 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:17:35.113514 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:17:35.113514 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:17:35.113514 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:17:35.114549 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:17:35.114549 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:17:35.114549 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Nov 02 07:17:35.114549 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Nov 02 07:17:35.114549 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Nov 02 07:17:35.114549 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Nov 02 07:17:35.114549 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Nov 02 07:17:35.114549 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Nov 02 07:17:35.114549 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Nov 02 07:17:35.114549 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Nov 02 07:17:35.114549 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 305/616] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Nov 2 07:17:34 2022] GET /identity/v3/users/swift => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:17:35.115148 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3a0b03-4205-416c-ab9d-f831fac9e0a1 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:35.141104 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bde269-25fa-4586-bcf6-8fdaa0de1837 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:35.141815 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bde269-25fa-4586-bcf6-8fdaa0de1837 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:35.142159 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bde269-25fa-4586-bcf6-8fdaa0de1837 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:35.142533 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bde269-25fa-4586-bcf6-8fdaa0de1837 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:35.143171 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bde269-25fa-4586-bcf6-8fdaa0de1837 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:35.144589 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bde269-25fa-4586-bcf6-8fdaa0de1837 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:35.153434 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-00bde269-25fa-4586-bcf6-8fdaa0de1837 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Nov 02 07:17:35.153434 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:17:35.153434 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Nov 02 07:17:35.153434 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Nov 02 07:17:35.153434 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Nov 02 07:17:35.153434 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Nov 02 07:17:35.153434 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Nov 02 07:17:35.153434 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:17:35.153434 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Nov 02 07:17:35.153434 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:17:35.153434 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 07:17:35.153434 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:17:35.153434 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:17:35.153434 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:17:35.153434 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:17:35.153434 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:17:35.153434 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:17:35.154447 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:17:35.154447 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:17:35.154447 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:17:35.154447 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:17:35.154447 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:17:35.154447 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:17:35.154447 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Nov 02 07:17:35.154447 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_service(service_id) Nov 02 07:17:35.154447 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Nov 02 07:17:35.154447 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Nov 02 07:17:35.154447 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:17:35.154447 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:17:35.154447 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:17:35.154447 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:17:35.154447 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:17:35.154447 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:17:35.155383 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:17:35.155383 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:17:35.155383 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:17:35.155383 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:17:35.155383 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:17:35.155383 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:17:35.155383 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:17:35.155383 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:17:35.155383 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:17:35.155383 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:17:35.155383 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Nov 02 07:17:35.155383 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:17:35.155383 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Nov 02 07:17:35.155383 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:17:35.155383 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 312/617] 10.222.0.18 () {56 vars in 1129 bytes} [Wed Nov 2 07:17:34 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:17:35.156221 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35186b55-c1f7-4d27-b4ce-e20d705e8006 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:35.156221 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35186b55-c1f7-4d27-b4ce-e20d705e8006 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:35.156442 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35186b55-c1f7-4d27-b4ce-e20d705e8006 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:35.162500 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 313/618] 10.222.0.18 () {54 vars in 927 bytes} [Wed Nov 2 07:17:35 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:17:35.163281 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35186b55-c1f7-4d27-b4ce-e20d705e8006 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:35.220632 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:35.225410 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3a0b03-4205-416c-ab9d-f831fac9e0a1 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:35.259379 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212cb995-6bc4-49a5-9773-4d6ae883bd9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:35.260023 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212cb995-6bc4-49a5-9773-4d6ae883bd9f None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:35.260300 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212cb995-6bc4-49a5-9773-4d6ae883bd9f None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:35.260575 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212cb995-6bc4-49a5-9773-4d6ae883bd9f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:35.262013 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35186b55-c1f7-4d27-b4ce-e20d705e8006 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:35.266968 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212cb995-6bc4-49a5-9773-4d6ae883bd9f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:35.268385 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212cb995-6bc4-49a5-9773-4d6ae883bd9f None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:35.279984 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-212cb995-6bc4-49a5-9773-4d6ae883bd9f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 07:17:35.279984 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:17:35.279984 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:17:35.279984 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:17:35.279984 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:17:35.279984 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:17:35.279984 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:17:35.279984 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:17:35.279984 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:17:35.279984 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:17:35.279984 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:17:35.279984 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:17:35.279984 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Nov 02 07:17:35.279984 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_project(project_id) Nov 02 07:17:35.279984 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Nov 02 07:17:35.281013 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Nov 02 07:17:35.281013 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:17:35.281013 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:17:35.281013 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:17:35.281013 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:17:35.281013 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:17:35.281013 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:17:35.281013 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:17:35.281013 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Nov 02 07:17:35.281013 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:17:35.281013 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:17:35.281013 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:17:35.281013 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:17:35.281013 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:17:35.281013 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:17:35.281013 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:17:35.281013 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:17:35.281904 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Nov 02 07:17:35.281904 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Nov 02 07:17:35.281904 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Nov 02 07:17:35.281904 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Nov 02 07:17:35.281904 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Nov 02 07:17:35.281904 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Nov 02 07:17:35.281904 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 07:17:35.281904 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Nov 02 07:17:35.281904 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 306/619] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Nov 2 07:17:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:17:35.281904 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212cb995-6bc4-49a5-9773-4d6ae883bd9f None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:35.300203 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada3236c-ae78-44e6-b7f1-be208942ddac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:35.300735 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada3236c-ae78-44e6-b7f1-be208942ddac None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:35.301009 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada3236c-ae78-44e6-b7f1-be208942ddac None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:35.301319 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada3236c-ae78-44e6-b7f1-be208942ddac None admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:35.302037 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada3236c-ae78-44e6-b7f1-be208942ddac None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:17:35.302329 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada3236c-ae78-44e6-b7f1-be208942ddac None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:35.303754 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada3236c-ae78-44e6-b7f1-be208942ddac None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:35.320435 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 314/620] 10.222.0.18 () {56 vars in 1135 bytes} [Wed Nov 2 07:17:35 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:17:35.321612 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a530bcc-6861-45dd-b8f6-0d0fdffc6abb None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:35.321955 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a530bcc-6861-45dd-b8f6-0d0fdffc6abb None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:35.322231 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a530bcc-6861-45dd-b8f6-0d0fdffc6abb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:35.340106 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-6a530bcc-6861-45dd-b8f6-0d0fdffc6abb None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:17:35.341271 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-6a530bcc-6861-45dd-b8f6-0d0fdffc6abb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:17:35.342248 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:35.383764 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212cb995-6bc4-49a5-9773-4d6ae883bd9f None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:35.406482 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 315/621] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Nov 2 07:17:35 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 07:17:35.407300 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a530bcc-6861-45dd-b8f6-0d0fdffc6abb None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:35.416128 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1472c6-4d46-4214-b72e-5b13953c466b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:35.416732 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1472c6-4d46-4214-b72e-5b13953c466b None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:35.417011 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1472c6-4d46-4214-b72e-5b13953c466b None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:35.417943 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1472c6-4d46-4214-b72e-5b13953c466b None admin] PATH_INFO: `/v3/services` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:35.417943 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1472c6-4d46-4214-b72e-5b13953c466b None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:17:35.418165 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1472c6-4d46-4214-b72e-5b13953c466b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:35.419553 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1472c6-4d46-4214-b72e-5b13953c466b None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:35.429187 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 307/622] 10.222.0.18 () {56 vars in 1124 bytes} [Wed Nov 2 07:17:35 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:17:35.429187 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1472c6-4d46-4214-b72e-5b13953c466b None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:35.470696 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:35.526319 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a530bcc-6861-45dd-b8f6-0d0fdffc6abb None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:35.533061 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1472c6-4d46-4214-b72e-5b13953c466b None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:35.561172 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:35.561318 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d17bc4-494a-4f66-994e-97519fb66bbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:35.561904 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d17bc4-494a-4f66-994e-97519fb66bbe None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:35.562113 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d17bc4-494a-4f66-994e-97519fb66bbe None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:35.562427 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d17bc4-494a-4f66-994e-97519fb66bbe None admin] PATH_INFO: `/v3/services/network` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:35.563015 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d17bc4-494a-4f66-994e-97519fb66bbe None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:35.564480 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d17bc4-494a-4f66-994e-97519fb66bbe None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:35.573299 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-f4d17bc4-494a-4f66-994e-97519fb66bbe None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Nov 02 07:17:35.573299 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:17:35.573299 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Nov 02 07:17:35.573299 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Nov 02 07:17:35.573299 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Nov 02 07:17:35.573299 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Nov 02 07:17:35.573299 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Nov 02 07:17:35.573299 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:17:35.573299 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Nov 02 07:17:35.573299 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:17:35.573299 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 07:17:35.573299 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:17:35.573299 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:17:35.573299 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:17:35.573299 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:17:35.573299 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:17:35.573299 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:17:35.574278 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:17:35.574278 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:17:35.574278 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:17:35.574278 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:17:35.574278 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:17:35.574278 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:17:35.574278 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Nov 02 07:17:35.574278 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_service(service_id) Nov 02 07:17:35.574278 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Nov 02 07:17:35.574278 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Nov 02 07:17:35.574278 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:17:35.574278 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:17:35.574278 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:17:35.574278 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:17:35.574278 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:17:35.574278 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:17:35.575717 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:17:35.575717 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:17:35.575717 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:17:35.575717 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:17:35.575717 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:17:35.575717 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:17:35.575717 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:17:35.575717 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:17:35.575717 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:17:35.575717 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:17:35.575717 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Nov 02 07:17:35.575717 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:17:35.575717 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Nov 02 07:17:35.575717 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:17:35.576467 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b4520f-3fc6-4192-9119-1e8ae0153caa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:35.576467 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b4520f-3fc6-4192-9119-1e8ae0153caa None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:35.576467 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b4520f-3fc6-4192-9119-1e8ae0153caa None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:35.576467 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b4520f-3fc6-4192-9119-1e8ae0153caa None admin] PATH_INFO: `/v3/projects` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:35.576738 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b4520f-3fc6-4192-9119-1e8ae0153caa None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:17:35.576738 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b4520f-3fc6-4192-9119-1e8ae0153caa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:35.577388 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b4520f-3fc6-4192-9119-1e8ae0153caa None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:35.579142 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 316/623] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Nov 2 07:17:35 2022] GET /identity/v3/services/network => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:17:35.579923 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d17bc4-494a-4f66-994e-97519fb66bbe None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:35.596217 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 308/624] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Nov 2 07:17:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:17:35.596217 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b4520f-3fc6-4192-9119-1e8ae0153caa None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:35.694918 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d17bc4-494a-4f66-994e-97519fb66bbe None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:35.699424 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b4520f-3fc6-4192-9119-1e8ae0153caa None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:35.707123 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:35.729344 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c496bfe-3085-402d-86d0-26e61a4b4460 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:35.729977 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c496bfe-3085-402d-86d0-26e61a4b4460 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:35.730257 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c496bfe-3085-402d-86d0-26e61a4b4460 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:35.730587 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c496bfe-3085-402d-86d0-26e61a4b4460 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:35.733301 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54afdd49-6b0f-4df5-a1cb-bd222cf2eb61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:35.733894 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54afdd49-6b0f-4df5-a1cb-bd222cf2eb61 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:35.734168 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54afdd49-6b0f-4df5-a1cb-bd222cf2eb61 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:35.734492 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54afdd49-6b0f-4df5-a1cb-bd222cf2eb61 None admin] PATH_INFO: `/v3/services` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:35.735102 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54afdd49-6b0f-4df5-a1cb-bd222cf2eb61 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:17:35.735394 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54afdd49-6b0f-4df5-a1cb-bd222cf2eb61 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:35.736733 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54afdd49-6b0f-4df5-a1cb-bd222cf2eb61 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:35.741611 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 309/625] 10.222.0.18 () {56 vars in 1124 bytes} [Wed Nov 2 07:17:35 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:17:35.742520 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54afdd49-6b0f-4df5-a1cb-bd222cf2eb61 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:35.790602 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:35.797740 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c496bfe-3085-402d-86d0-26e61a4b4460 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:35.799210 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c496bfe-3085-402d-86d0-26e61a4b4460 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:35.843228 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54afdd49-6b0f-4df5-a1cb-bd222cf2eb61 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:35.878069 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12238e3e-2130-4921-80e9-b263de8c1cb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:35.878841 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12238e3e-2130-4921-80e9-b263de8c1cb8 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:35.879124 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12238e3e-2130-4921-80e9-b263de8c1cb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:35.880017 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12238e3e-2130-4921-80e9-b263de8c1cb8 None admin] PATH_INFO: `/v3/services` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:35.880371 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12238e3e-2130-4921-80e9-b263de8c1cb8 None admin] RBAC: Adding query filter params (name=network) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:17:35.880644 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12238e3e-2130-4921-80e9-b263de8c1cb8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:35.882018 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12238e3e-2130-4921-80e9-b263de8c1cb8 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:35.886336 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 310/626] 10.222.0.18 () {56 vars in 1112 bytes} [Wed Nov 2 07:17:35 2022] GET /identity/v3/services?name=network => generated 124 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:17:35.887215 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12238e3e-2130-4921-80e9-b263de8c1cb8 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:35.898305 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 317/627] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Nov 2 07:17:35 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Nov 02 07:17:35.899141 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c496bfe-3085-402d-86d0-26e61a4b4460 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:35.913318 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:35.991174 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12238e3e-2130-4921-80e9-b263de8c1cb8 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:35.998389 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c496bfe-3085-402d-86d0-26e61a4b4460 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:36.001416 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:36.025751 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1deae9-9e75-46ff-915b-e6f249315ab8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:36.026344 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1deae9-9e75-46ff-915b-e6f249315ab8 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:36.026713 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1deae9-9e75-46ff-915b-e6f249315ab8 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:36.026984 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1deae9-9e75-46ff-915b-e6f249315ab8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:36.027692 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1deae9-9e75-46ff-915b-e6f249315ab8 None admin] RBAC: Adding query filter params (role.id=4af6f4c514284b8cbddf3e02ec826378, scope.project.id=af04daa468b64160a6176dcab29502a7, user.id=ff4736f3d50746b0bf5d6a917b2ca0cf) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:17:36.027995 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1deae9-9e75-46ff-915b-e6f249315ab8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:36.029399 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1deae9-9e75-46ff-915b-e6f249315ab8 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:36.031417 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7613f6b-50d6-4646-a456-a5e540fd70f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:36.032021 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7613f6b-50d6-4646-a456-a5e540fd70f3 None admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:36.032294 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7613f6b-50d6-4646-a456-a5e540fd70f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:36.032556 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7613f6b-50d6-4646-a456-a5e540fd70f3 None admin] PATH_INFO: `/v3/services` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:36.033193 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7613f6b-50d6-4646-a456-a5e540fd70f3 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:36.034550 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7613f6b-50d6-4646-a456-a5e540fd70f3 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:36.034974 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 311/628] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Nov 2 07:17:35 2022] GET /identity/v3/role_assignments?user.id=ff4736f3d50746b0bf5d6a917b2ca0cf&scope.project.id=af04daa468b64160a6176dcab29502a7&role.id=4af6f4c514284b8cbddf3e02ec826378 => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:17:36.035785 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1deae9-9e75-46ff-915b-e6f249315ab8 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:36.066420 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 318/629] 10.222.0.18 () {60 vars in 1141 bytes} [Wed Nov 2 07:17:35 2022] POST /identity/v3/services => generated 269 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:17:36.066420 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7613f6b-50d6-4646-a456-a5e540fd70f3 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:36.171412 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1deae9-9e75-46ff-915b-e6f249315ab8 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:36.187889 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:36.240807 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acd2371-173e-4023-8e69-0cfd7b0e87d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:36.241354 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acd2371-173e-4023-8e69-0cfd7b0e87d6 None admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:36.241354 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acd2371-173e-4023-8e69-0cfd7b0e87d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:36.241629 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acd2371-173e-4023-8e69-0cfd7b0e87d6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:36.242311 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acd2371-173e-4023-8e69-0cfd7b0e87d6 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:36.243784 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acd2371-173e-4023-8e69-0cfd7b0e87d6 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:36.244574 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7613f6b-50d6-4646-a456-a5e540fd70f3 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:36.272967 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 312/630] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Nov 2 07:17:36 2022] POST /identity/v3/endpoints => generated 346 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:17:36.274050 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acd2371-173e-4023-8e69-0cfd7b0e87d6 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:36.304895 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fab3abd-2afd-4e46-9d6f-6946067189ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:36.305785 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fab3abd-2afd-4e46-9d6f-6946067189ac None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:36.306261 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fab3abd-2afd-4e46-9d6f-6946067189ac None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:36.306765 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fab3abd-2afd-4e46-9d6f-6946067189ac None admin] PATH_INFO: `/v3/services` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:36.307583 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fab3abd-2afd-4e46-9d6f-6946067189ac None admin] RBAC: Adding query filter params (type=network) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:17:36.309231 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fab3abd-2afd-4e46-9d6f-6946067189ac None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:36.309368 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fab3abd-2afd-4e46-9d6f-6946067189ac None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:36.313298 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:36.316587 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 319/631] 10.222.0.18 () {56 vars in 1112 bytes} [Wed Nov 2 07:17:36 2022] GET /identity/v3/services?type=network => generated 350 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:17:36.323202 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fab3abd-2afd-4e46-9d6f-6946067189ac None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:36.439842 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acd2371-173e-4023-8e69-0cfd7b0e87d6 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:36.468376 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fab3abd-2afd-4e46-9d6f-6946067189ac None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:36.502777 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:36.503902 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e767d1c4-34b9-4627-9949-5c83132c462c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:36.505461 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e767d1c4-34b9-4627-9949-5c83132c462c None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:36.506091 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e767d1c4-34b9-4627-9949-5c83132c462c None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:36.506722 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e767d1c4-34b9-4627-9949-5c83132c462c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:36.512310 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba047960-ed99-4eb2-9370-16ac204ef1f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:36.512919 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba047960-ed99-4eb2-9370-16ac204ef1f7 None admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:36.513200 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba047960-ed99-4eb2-9370-16ac204ef1f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:36.513472 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba047960-ed99-4eb2-9370-16ac204ef1f7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:36.514151 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba047960-ed99-4eb2-9370-16ac204ef1f7 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:36.516351 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba047960-ed99-4eb2-9370-16ac204ef1f7 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:36.522106 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e767d1c4-34b9-4627-9949-5c83132c462c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:36.524526 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e767d1c4-34b9-4627-9949-5c83132c462c None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:36.541668 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-e767d1c4-34b9-4627-9949-5c83132c462c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:17:36.541668 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:17:36.541668 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:17:36.541668 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:17:36.541668 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:17:36.541668 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:17:36.541668 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:17:36.541668 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:17:36.541668 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:17:36.541668 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:17:36.541668 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:17:36.541668 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:17:36.541668 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:17:36.541668 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:17:36.541668 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:17:36.541668 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:17:36.542852 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:17:36.542852 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:17:36.542852 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:17:36.542852 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:17:36.542852 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:17:36.542852 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:17:36.542852 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:17:36.542852 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Nov 02 07:17:36.542852 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:17:36.542852 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:17:36.542852 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:17:36.542852 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:17:36.542852 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:17:36.542852 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:17:36.542852 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:17:36.542852 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:17:36.542852 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:17:36.543812 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:17:36.543812 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:17:36.543812 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Nov 02 07:17:36.545659 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 313/632] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Nov 2 07:17:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:17:36.547301 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 320/633] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Nov 2 07:17:36 2022] POST /identity/v3/endpoints => generated 337 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:17:36.551462 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e767d1c4-34b9-4627-9949-5c83132c462c None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:36.663778 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:36.689357 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e767d1c4-34b9-4627-9949-5c83132c462c None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:36.739574 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9bb53c-b65c-4db0-b14b-fbb1252e64de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:36.740109 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9bb53c-b65c-4db0-b14b-fbb1252e64de None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:36.740442 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9bb53c-b65c-4db0-b14b-fbb1252e64de None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:36.740715 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9bb53c-b65c-4db0-b14b-fbb1252e64de None admin] PATH_INFO: `/v3/domains` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:36.741357 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9bb53c-b65c-4db0-b14b-fbb1252e64de None admin] RBAC: Adding query filter params (name=Default) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:17:36.741721 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9bb53c-b65c-4db0-b14b-fbb1252e64de None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:36.743069 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9bb53c-b65c-4db0-b14b-fbb1252e64de None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:36.760294 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 314/634] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Nov 2 07:17:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:17:36.761674 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba047960-ed99-4eb2-9370-16ac204ef1f7 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:36.873278 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba047960-ed99-4eb2-9370-16ac204ef1f7 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:36.910477 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85673644-bf90-4e53-b664-2c9b027a47ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:36.911681 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85673644-bf90-4e53-b664-2c9b027a47ae None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:36.912176 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85673644-bf90-4e53-b664-2c9b027a47ae None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:36.912654 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85673644-bf90-4e53-b664-2c9b027a47ae None admin] PATH_INFO: `/v3/projects/service` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:36.920206 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85673644-bf90-4e53-b664-2c9b027a47ae None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:36.922040 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85673644-bf90-4e53-b664-2c9b027a47ae None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:36.936352 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-85673644-bf90-4e53-b664-2c9b027a47ae None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 07:17:36.936352 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:17:36.936352 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:17:36.936352 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:17:36.936352 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:17:36.936352 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:17:36.936352 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:17:36.936352 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:17:36.936352 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:17:36.936352 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:17:36.936352 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:17:36.936352 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:17:36.936352 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Nov 02 07:17:36.936352 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_project(project_id) Nov 02 07:17:36.936352 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Nov 02 07:17:36.937295 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Nov 02 07:17:36.937295 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:17:36.937295 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:17:36.937295 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:17:36.937295 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:17:36.937295 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:17:36.937295 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:17:36.937295 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:17:36.937295 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:17:36.937295 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:17:36.937295 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:17:36.937295 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:17:36.937295 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:17:36.937295 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:17:36.937295 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:17:36.937295 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:17:36.937295 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:17:36.938163 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Nov 02 07:17:36.938163 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Nov 02 07:17:36.938163 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Nov 02 07:17:36.938163 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Nov 02 07:17:36.938163 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Nov 02 07:17:36.938163 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Nov 02 07:17:36.938163 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 07:17:36.938163 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:17:36.940539 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 321/635] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Nov 2 07:17:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:17:36.946757 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9bb53c-b65c-4db0-b14b-fbb1252e64de None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:37.078980 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9bb53c-b65c-4db0-b14b-fbb1252e64de None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:37.116070 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9f1860-3e67-4d34-8f06-18ac4484b596 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:37.116681 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9f1860-3e67-4d34-8f06-18ac4484b596 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:37.116958 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9f1860-3e67-4d34-8f06-18ac4484b596 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:37.117227 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9f1860-3e67-4d34-8f06-18ac4484b596 None admin] PATH_INFO: `/v3/projects` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:37.117909 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9f1860-3e67-4d34-8f06-18ac4484b596 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:17:37.118208 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9f1860-3e67-4d34-8f06-18ac4484b596 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:37.119725 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9f1860-3e67-4d34-8f06-18ac4484b596 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:37.121360 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:37.136293 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 315/636] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Nov 2 07:17:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:17:37.144937 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85673644-bf90-4e53-b664-2c9b027a47ae None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:37.194742 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:37.253340 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85673644-bf90-4e53-b664-2c9b027a47ae None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:37.289532 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566232e5-87a7-448d-a27c-b982f7f92d58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:37.290516 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566232e5-87a7-448d-a27c-b982f7f92d58 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:37.290975 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566232e5-87a7-448d-a27c-b982f7f92d58 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:37.291423 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566232e5-87a7-448d-a27c-b982f7f92d58 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:37.292685 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566232e5-87a7-448d-a27c-b982f7f92d58 None admin] RBAC: Adding query filter params (role.id=c4bf019fc6c44c71a6c984f6d0bab75f, scope.project.id=af04daa468b64160a6176dcab29502a7, user.id=f9116f47889f4db39e0203d6cdae1593) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:17:37.293189 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566232e5-87a7-448d-a27c-b982f7f92d58 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:37.295700 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566232e5-87a7-448d-a27c-b982f7f92d58 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:37.304150 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 322/637] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Nov 2 07:17:37 2022] GET /identity/v3/role_assignments?user.id=f9116f47889f4db39e0203d6cdae1593&scope.project.id=af04daa468b64160a6176dcab29502a7&role.id=c4bf019fc6c44c71a6c984f6d0bab75f => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:17:38.463189 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e15975-5643-48e8-9fc9-c0b6d9ee2981 None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:38.463716 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e15975-5643-48e8-9fc9-c0b6d9ee2981 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:38.463716 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e15975-5643-48e8-9fc9-c0b6d9ee2981 None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:38.465373 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 316/638] 10.222.0.18 () {54 vars in 927 bytes} [Wed Nov 2 07:17:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:17:38.470747 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c19f3bb-5637-4342-9986-4f3b48f2932b None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:38.471175 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c19f3bb-5637-4342-9986-4f3b48f2932b None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:38.471349 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c19f3bb-5637-4342-9986-4f3b48f2932b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:38.490252 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-6c19f3bb-5637-4342-9986-4f3b48f2932b None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:17:38.491458 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-6c19f3bb-5637-4342-9986-4f3b48f2932b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:17:38.518375 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c896b6b9-2b75-4005-a1da-38337e26b2cb None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:38.518681 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c896b6b9-2b75-4005-a1da-38337e26b2cb None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:38.518950 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c896b6b9-2b75-4005-a1da-38337e26b2cb None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:38.521711 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 317/639] 10.222.0.18 () {54 vars in 927 bytes} [Wed Nov 2 07:17:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Nov 02 07:17:38.527960 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f17cae-2fab-4341-8831-b2705414b7ad None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:38.528246 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f17cae-2fab-4341-8831-b2705414b7ad None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:38.528520 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f17cae-2fab-4341-8831-b2705414b7ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:38.552946 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-24f17cae-2fab-4341-8831-b2705414b7ad None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:17:38.554772 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-24f17cae-2fab-4341-8831-b2705414b7ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:17:38.570441 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 323/640] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Nov 2 07:17:38 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 07:17:38.625045 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 318/641] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Nov 2 07:17:38 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Nov 02 07:17:38.842778 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c19f3bb-5637-4342-9986-4f3b48f2932b None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:38.858399 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62187b81-635c-4196-ae53-a9476017b6c8 None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:38.858944 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62187b81-635c-4196-ae53-a9476017b6c8 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:38.859225 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62187b81-635c-4196-ae53-a9476017b6c8 None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:38.860788 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 319/642] 10.222.0.18 () {54 vars in 927 bytes} [Wed Nov 2 07:17:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:17:38.866699 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b3f733-24b9-4041-a5fd-c5da65bb5d27 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:38.867233 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b3f733-24b9-4041-a5fd-c5da65bb5d27 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:38.867692 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b3f733-24b9-4041-a5fd-c5da65bb5d27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:38.885286 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:38.888679 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-40b3f733-24b9-4041-a5fd-c5da65bb5d27 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:17:38.890102 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-40b3f733-24b9-4041-a5fd-c5da65bb5d27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:17:38.956379 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c19f3bb-5637-4342-9986-4f3b48f2932b None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:38.972155 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 320/643] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Nov 2 07:17:38 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Nov 02 07:17:38.975368 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b3f733-24b9-4041-a5fd-c5da65bb5d27 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:38.994662 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a524a103-0725-4445-bb2c-b8ec0e87d95f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:38.995379 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a524a103-0725-4445-bb2c-b8ec0e87d95f None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:38.995721 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a524a103-0725-4445-bb2c-b8ec0e87d95f None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:38.996056 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a524a103-0725-4445-bb2c-b8ec0e87d95f None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:39.007279 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a524a103-0725-4445-bb2c-b8ec0e87d95f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:17:39.013712 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-a524a103-0725-4445-bb2c-b8ec0e87d95f None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Nov 02 07:17:39.013712 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:17:39.013712 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:17:39.013712 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:17:39.013712 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:17:39.013712 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:17:39.013712 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:17:39.013712 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:17:39.013712 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:17:39.013712 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:17:39.013712 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:17:39.013712 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:17:39.013712 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:17:39.013712 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:17:39.013712 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:17:39.013712 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:17:39.014703 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:17:39.014703 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:17:39.014703 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:17:39.014703 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:17:39.014703 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:17:39.014703 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:17:39.014703 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:17:39.014703 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:17:39.014703 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:17:39.014703 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:17:39.014703 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:17:39.014703 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:17:39.014703 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:17:39.014703 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:17:39.014703 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:17:39.014703 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:17:39.020974 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:17:39.020974 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:17:39.020974 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:17:39.020974 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:17:39.020974 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:17:39.020974 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:17:39.020974 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:17:39.020974 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:17:39.020974 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:17:39.020974 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:17:39.020974 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Nov 02 07:17:39.020974 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:17:39.020974 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 324/644] 10.222.0.18 () {56 vars in 1120 bytes} [Wed Nov 2 07:17:38 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:17:39.025153 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a524a103-0725-4445-bb2c-b8ec0e87d95f None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:39.063876 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:39.122906 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b3f733-24b9-4041-a5fd-c5da65bb5d27 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:39.133257 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:39.138812 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a524a103-0725-4445-bb2c-b8ec0e87d95f None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:39.168286 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50220e5a-74af-43d2-a16b-4dc8c7644c76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:39.169439 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50220e5a-74af-43d2-a16b-4dc8c7644c76 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:39.169876 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50220e5a-74af-43d2-a16b-4dc8c7644c76 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:39.170454 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50220e5a-74af-43d2-a16b-4dc8c7644c76 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:39.171482 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50220e5a-74af-43d2-a16b-4dc8c7644c76 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:39.173548 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50220e5a-74af-43d2-a16b-4dc8c7644c76 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:39.176000 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25124584-71da-4249-8c9c-623fc48b15c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:39.177416 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25124584-71da-4249-8c9c-623fc48b15c6 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:39.177786 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25124584-71da-4249-8c9c-623fc48b15c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:39.178151 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25124584-71da-4249-8c9c-623fc48b15c6 None admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:39.178963 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25124584-71da-4249-8c9c-623fc48b15c6 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:17:39.179321 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25124584-71da-4249-8c9c-623fc48b15c6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:39.180753 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25124584-71da-4249-8c9c-623fc48b15c6 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:39.185101 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-50220e5a-74af-43d2-a16b-4dc8c7644c76 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Nov 02 07:17:39.185101 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:17:39.185101 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Nov 02 07:17:39.185101 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Nov 02 07:17:39.185101 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Nov 02 07:17:39.185101 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Nov 02 07:17:39.185101 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Nov 02 07:17:39.185101 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:17:39.185101 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Nov 02 07:17:39.185101 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Nov 02 07:17:39.185101 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 07:17:39.185101 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Nov 02 07:17:39.185101 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:17:39.185101 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:17:39.185101 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:17:39.185101 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:17:39.186228 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:17:39.186228 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:17:39.186228 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:17:39.186228 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:17:39.186228 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:17:39.186228 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:17:39.186228 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:17:39.186228 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Nov 02 07:17:39.186228 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_service(service_id) Nov 02 07:17:39.186228 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Nov 02 07:17:39.186228 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Nov 02 07:17:39.186228 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:17:39.186228 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:17:39.186228 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:17:39.186228 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:17:39.186228 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:17:39.186228 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:17:39.187344 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:17:39.187344 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Nov 02 07:17:39.187344 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:17:39.187344 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:17:39.187344 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:17:39.187344 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:17:39.187344 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:17:39.187344 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:17:39.187344 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:17:39.187344 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:17:39.187344 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Nov 02 07:17:39.187344 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:17:39.187344 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Nov 02 07:17:39.187344 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Nov 02 07:17:39.189852 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 321/645] 10.222.0.18 () {56 vars in 1132 bytes} [Wed Nov 2 07:17:38 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:17:39.196541 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 325/646] 10.222.0.18 () {56 vars in 1115 bytes} [Wed Nov 2 07:17:39 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:17:39.197597 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50220e5a-74af-43d2-a16b-4dc8c7644c76 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:39.202932 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25124584-71da-4249-8c9c-623fc48b15c6 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:39.307855 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50220e5a-74af-43d2-a16b-4dc8c7644c76 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:39.319520 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25124584-71da-4249-8c9c-623fc48b15c6 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:39.327185 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:39.352464 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5866c4c3-85fc-4ba6-8ad0-20aea846f956 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:39.353337 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5866c4c3-85fc-4ba6-8ad0-20aea846f956 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:39.353641 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5866c4c3-85fc-4ba6-8ad0-20aea846f956 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:39.353886 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5866c4c3-85fc-4ba6-8ad0-20aea846f956 None admin] PATH_INFO: `/v3/services` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:39.354601 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5866c4c3-85fc-4ba6-8ad0-20aea846f956 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:17:39.354915 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5866c4c3-85fc-4ba6-8ad0-20aea846f956 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:39.356057 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacd5644-b534-4e88-848e-0fbd7015a875 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:39.356245 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5866c4c3-85fc-4ba6-8ad0-20aea846f956 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:39.356667 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacd5644-b534-4e88-848e-0fbd7015a875 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:39.357011 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacd5644-b534-4e88-848e-0fbd7015a875 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:39.357555 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacd5644-b534-4e88-848e-0fbd7015a875 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:39.365669 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 322/647] 10.222.0.18 () {56 vars in 1126 bytes} [Wed Nov 2 07:17:39 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:17:39.365669 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5866c4c3-85fc-4ba6-8ad0-20aea846f956 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:39.370229 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacd5644-b534-4e88-848e-0fbd7015a875 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:39.371705 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacd5644-b534-4e88-848e-0fbd7015a875 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:39.388553 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-bacd5644-b534-4e88-848e-0fbd7015a875 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:17:39.388553 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:17:39.388553 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:17:39.388553 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:17:39.388553 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:17:39.388553 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:17:39.388553 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:17:39.388553 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:17:39.388553 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:17:39.388553 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:17:39.388553 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:17:39.388553 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:17:39.388553 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:17:39.388553 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:17:39.388553 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:17:39.388553 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:17:39.389527 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:17:39.389527 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:17:39.389527 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:17:39.389527 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:17:39.389527 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:17:39.389527 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:17:39.389527 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:17:39.389527 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:17:39.389527 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:17:39.389527 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:17:39.389527 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:17:39.389527 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:17:39.389527 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:17:39.389527 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:17:39.389527 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:17:39.389527 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:17:39.389527 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:17:39.390472 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:17:39.390472 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:17:39.390472 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:17:39.390472 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 326/648] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Nov 2 07:17:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:17:39.391862 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacd5644-b534-4e88-848e-0fbd7015a875 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:39.410242 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:39.479341 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5866c4c3-85fc-4ba6-8ad0-20aea846f956 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:39.502385 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacd5644-b534-4e88-848e-0fbd7015a875 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:39.516262 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f341bb-9ebc-4a38-835d-28d3c2aea0a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:39.516887 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f341bb-9ebc-4a38-835d-28d3c2aea0a9 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:39.517162 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f341bb-9ebc-4a38-835d-28d3c2aea0a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:39.517425 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f341bb-9ebc-4a38-835d-28d3c2aea0a9 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:39.517992 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f341bb-9ebc-4a38-835d-28d3c2aea0a9 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:39.519385 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f341bb-9ebc-4a38-835d-28d3c2aea0a9 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:39.528016 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-c8f341bb-9ebc-4a38-835d-28d3c2aea0a9 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Nov 02 07:17:39.528016 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:17:39.528016 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Nov 02 07:17:39.528016 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Nov 02 07:17:39.528016 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Nov 02 07:17:39.528016 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Nov 02 07:17:39.528016 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Nov 02 07:17:39.528016 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:17:39.528016 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Nov 02 07:17:39.528016 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Nov 02 07:17:39.528016 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 07:17:39.528016 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Nov 02 07:17:39.528016 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:17:39.528016 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:17:39.528016 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:17:39.528016 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:17:39.528016 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:17:39.529042 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:17:39.529042 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:17:39.529042 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:17:39.529042 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:17:39.529042 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:17:39.529042 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:17:39.529042 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Nov 02 07:17:39.529042 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_service(service_id) Nov 02 07:17:39.529042 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Nov 02 07:17:39.529042 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Nov 02 07:17:39.529042 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:17:39.529042 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:17:39.529042 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:17:39.529042 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:17:39.529042 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:17:39.529042 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:17:39.529888 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:17:39.529888 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Nov 02 07:17:39.529888 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:17:39.529888 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:17:39.529888 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:17:39.529888 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:17:39.529888 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:17:39.529888 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:17:39.529888 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:17:39.529888 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:17:39.529888 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Nov 02 07:17:39.529888 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:17:39.529888 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Nov 02 07:17:39.529888 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Nov 02 07:17:39.529888 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 323/649] 10.222.0.18 () {56 vars in 1114 bytes} [Wed Nov 2 07:17:39 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:17:39.531259 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f341bb-9ebc-4a38-835d-28d3c2aea0a9 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:39.538661 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12656fb4-394b-46c3-8627-d69fbdada08e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:39.539267 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12656fb4-394b-46c3-8627-d69fbdada08e None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:39.539543 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12656fb4-394b-46c3-8627-d69fbdada08e None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:39.539808 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12656fb4-394b-46c3-8627-d69fbdada08e None admin] PATH_INFO: `/v3/services` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:39.540428 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12656fb4-394b-46c3-8627-d69fbdada08e None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:17:39.540722 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12656fb4-394b-46c3-8627-d69fbdada08e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:39.542067 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12656fb4-394b-46c3-8627-d69fbdada08e None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:39.546816 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 327/650] 10.222.0.18 () {56 vars in 1126 bytes} [Wed Nov 2 07:17:39 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:17:39.547942 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7056b818-00cb-47dd-91a7-62a055968d38 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:39.548224 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7056b818-00cb-47dd-91a7-62a055968d38 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:39.548496 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7056b818-00cb-47dd-91a7-62a055968d38 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:39.549666 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 328/651] 10.222.0.18 () {54 vars in 927 bytes} [Wed Nov 2 07:17:39 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:17:39.550354 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7056b818-00cb-47dd-91a7-62a055968d38 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:39.583635 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:39.629846 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:39.636544 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f341bb-9ebc-4a38-835d-28d3c2aea0a9 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:39.654912 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7056b818-00cb-47dd-91a7-62a055968d38 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:39.672526 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7574b167-dc2f-4a02-8e66-23589d4dac3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:39.673122 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7574b167-dc2f-4a02-8e66-23589d4dac3d None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:39.673398 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7574b167-dc2f-4a02-8e66-23589d4dac3d None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:39.673664 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7574b167-dc2f-4a02-8e66-23589d4dac3d None admin] PATH_INFO: `/v3/domains` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:39.674287 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7574b167-dc2f-4a02-8e66-23589d4dac3d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:17:39.674653 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7574b167-dc2f-4a02-8e66-23589d4dac3d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:39.676015 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7574b167-dc2f-4a02-8e66-23589d4dac3d None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:39.689977 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 324/652] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Nov 2 07:17:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:17:39.690828 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7574b167-dc2f-4a02-8e66-23589d4dac3d None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:39.691804 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28175429-a654-44bc-ac15-bcf9e22b2f2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:39.692576 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28175429-a654-44bc-ac15-bcf9e22b2f2d None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:39.692858 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28175429-a654-44bc-ac15-bcf9e22b2f2d None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:39.693128 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28175429-a654-44bc-ac15-bcf9e22b2f2d None admin] PATH_INFO: `/v3/services` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:39.693745 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28175429-a654-44bc-ac15-bcf9e22b2f2d None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:17:39.694053 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28175429-a654-44bc-ac15-bcf9e22b2f2d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:39.695512 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28175429-a654-44bc-ac15-bcf9e22b2f2d None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:39.704473 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 329/653] 10.222.0.18 () {56 vars in 1114 bytes} [Wed Nov 2 07:17:39 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:17:39.704473 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c769467-01d5-49c2-99c7-49c2bfd683b5 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:39.704473 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c769467-01d5-49c2-99c7-49c2bfd683b5 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:39.704473 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c769467-01d5-49c2-99c7-49c2bfd683b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:39.720081 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-0c769467-01d5-49c2-99c7-49c2bfd683b5 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:17:39.721263 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-0c769467-01d5-49c2-99c7-49c2bfd683b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:17:39.791841 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 330/654] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Nov 2 07:17:39 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 07:17:39.792678 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c769467-01d5-49c2-99c7-49c2bfd683b5 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:39.801397 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7574b167-dc2f-4a02-8e66-23589d4dac3d None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:39.811299 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:39.840534 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a77dff-63ee-4713-8051-9995a5917e21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:39.841141 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a77dff-63ee-4713-8051-9995a5917e21 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:39.841416 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a77dff-63ee-4713-8051-9995a5917e21 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:39.841681 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a77dff-63ee-4713-8051-9995a5917e21 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:39.842412 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a77dff-63ee-4713-8051-9995a5917e21 None admin] RBAC: Adding query filter params (interface=public, service_id=a09ed7189f0b47a28ddc95b50ca67d94, region_id=RegionOne) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:17:39.842717 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a77dff-63ee-4713-8051-9995a5917e21 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:39.844062 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a77dff-63ee-4713-8051-9995a5917e21 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:39.853091 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 325/655] 10.222.0.18 () {56 vars in 1251 bytes} [Wed Nov 2 07:17:39 2022] GET /identity/v3/endpoints?service_id=a09ed7189f0b47a28ddc95b50ca67d94&interface=public®ion_id=RegionOne => generated 194 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:17:39.853632 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a77dff-63ee-4713-8051-9995a5917e21 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:39.875221 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:39.914164 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c769467-01d5-49c2-99c7-49c2bfd683b5 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:39.952445 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76a5ce9-634f-4319-acb2-bb5205d8e3b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:39.953107 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76a5ce9-634f-4319-acb2-bb5205d8e3b3 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:39.953387 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76a5ce9-634f-4319-acb2-bb5205d8e3b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:39.953656 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76a5ce9-634f-4319-acb2-bb5205d8e3b3 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:39.962012 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76a5ce9-634f-4319-acb2-bb5205d8e3b3 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:39.963481 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76a5ce9-634f-4319-acb2-bb5205d8e3b3 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:39.978748 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-e76a5ce9-634f-4319-acb2-bb5205d8e3b3 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Nov 02 07:17:39.978748 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:17:39.978748 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:17:39.978748 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:17:39.978748 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:17:39.978748 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:17:39.978748 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:17:39.978748 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:17:39.978748 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:17:39.978748 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:17:39.978748 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:17:39.978748 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:17:39.978748 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Nov 02 07:17:39.978748 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_user(user_id) Nov 02 07:17:39.978748 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Nov 02 07:17:39.978748 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Nov 02 07:17:39.979760 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:17:39.979760 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:17:39.979760 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Nov 02 07:17:39.979760 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:17:39.979760 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Nov 02 07:17:39.979760 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:17:39.979760 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:17:39.979760 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:17:39.979760 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:17:39.979760 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:17:39.979760 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:17:39.979760 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:17:39.979760 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:17:39.979760 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:17:39.979760 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:17:39.979760 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:17:39.979760 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:17:39.979760 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:17:39.980718 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:17:39.980718 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:17:39.980718 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Nov 02 07:17:39.980718 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Nov 02 07:17:39.980718 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Nov 02 07:17:39.980718 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Nov 02 07:17:39.980718 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Nov 02 07:17:39.980718 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Nov 02 07:17:39.980718 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Nov 02 07:17:39.980718 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:17:39.980718 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 331/656] 10.222.0.18 () {56 vars in 1117 bytes} [Wed Nov 2 07:17:39 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:17:39.980718 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76a5ce9-634f-4319-acb2-bb5205d8e3b3 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:39.995273 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a77dff-63ee-4713-8051-9995a5917e21 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:40.050151 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ccae05-9338-4239-85b2-fa8f12a1c355 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:40.051389 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ccae05-9338-4239-85b2-fa8f12a1c355 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:40.051916 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ccae05-9338-4239-85b2-fa8f12a1c355 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:40.052284 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ccae05-9338-4239-85b2-fa8f12a1c355 None admin] PATH_INFO: `/v3/services` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:40.053315 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ccae05-9338-4239-85b2-fa8f12a1c355 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:17:40.054149 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ccae05-9338-4239-85b2-fa8f12a1c355 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:40.055843 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ccae05-9338-4239-85b2-fa8f12a1c355 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:40.066644 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 326/657] 10.222.0.18 () {56 vars in 1114 bytes} [Wed Nov 2 07:17:39 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:17:40.066644 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ccae05-9338-4239-85b2-fa8f12a1c355 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:40.115137 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76a5ce9-634f-4319-acb2-bb5205d8e3b3 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:40.120363 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:40.160195 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:40.170214 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff6cbb8-ebca-4c84-94f5-42db019686b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:40.170787 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff6cbb8-ebca-4c84-94f5-42db019686b0 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:40.171070 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff6cbb8-ebca-4c84-94f5-42db019686b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:40.171339 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff6cbb8-ebca-4c84-94f5-42db019686b0 None admin] PATH_INFO: `/v3/services` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:40.171948 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff6cbb8-ebca-4c84-94f5-42db019686b0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:40.173961 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff6cbb8-ebca-4c84-94f5-42db019686b0 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:40.181094 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 332/658] 10.222.0.18 () {56 vars in 1087 bytes} [Wed Nov 2 07:17:39 2022] GET /identity/v3/services => generated 1029 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:17:40.181965 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff6cbb8-ebca-4c84-94f5-42db019686b0 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:40.222200 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ccae05-9338-4239-85b2-fa8f12a1c355 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:40.268085 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e278094b-5772-479b-bbb8-2b17c9247183 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:40.268748 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e278094b-5772-479b-bbb8-2b17c9247183 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:40.269076 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e278094b-5772-479b-bbb8-2b17c9247183 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:40.269350 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e278094b-5772-479b-bbb8-2b17c9247183 None admin] PATH_INFO: `/v3/users` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:40.270252 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e278094b-5772-479b-bbb8-2b17c9247183 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:17:40.270425 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e278094b-5772-479b-bbb8-2b17c9247183 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:40.271959 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e278094b-5772-479b-bbb8-2b17c9247183 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:40.300758 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 327/659] 10.222.0.18 () {56 vars in 1149 bytes} [Wed Nov 2 07:17:40 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:17:40.312130 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e278094b-5772-479b-bbb8-2b17c9247183 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:40.352196 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff6cbb8-ebca-4c84-94f5-42db019686b0 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:40.402168 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce773c6-1c79-4852-8910-63a874bc9c89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:40.402843 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce773c6-1c79-4852-8910-63a874bc9c89 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:40.403117 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce773c6-1c79-4852-8910-63a874bc9c89 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:40.404482 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce773c6-1c79-4852-8910-63a874bc9c89 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:40.404482 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce773c6-1c79-4852-8910-63a874bc9c89 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:40.405662 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce773c6-1c79-4852-8910-63a874bc9c89 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:40.421453 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-cce773c6-1c79-4852-8910-63a874bc9c89 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Nov 02 07:17:40.421453 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:17:40.421453 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Nov 02 07:17:40.421453 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Nov 02 07:17:40.421453 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Nov 02 07:17:40.421453 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Nov 02 07:17:40.421453 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Nov 02 07:17:40.421453 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:17:40.421453 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Nov 02 07:17:40.421453 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:17:40.421453 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 07:17:40.421453 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:17:40.421453 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:17:40.421453 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:17:40.421453 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:17:40.421453 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:17:40.421453 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:17:40.422558 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:17:40.422558 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:17:40.422558 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:17:40.422558 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:17:40.422558 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:17:40.422558 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:17:40.422558 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Nov 02 07:17:40.422558 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_service(service_id) Nov 02 07:17:40.422558 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Nov 02 07:17:40.422558 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Nov 02 07:17:40.422558 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:17:40.422558 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:17:40.422558 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:17:40.422558 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:17:40.422558 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:17:40.422558 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:17:40.424135 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:17:40.424135 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:17:40.424135 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:17:40.424135 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:17:40.424135 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:17:40.424135 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:17:40.424135 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:17:40.424135 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:17:40.424135 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:17:40.424135 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:17:40.424135 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Nov 02 07:17:40.424135 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:17:40.424135 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Nov 02 07:17:40.424135 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:17:40.424135 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 333/660] 10.222.0.18 () {56 vars in 1126 bytes} [Wed Nov 2 07:17:40 2022] GET /identity/v3/services/object-store => generated 93 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:17:40.430704 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce773c6-1c79-4852-8910-63a874bc9c89 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:40.467173 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:40.480836 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e278094b-5772-479b-bbb8-2b17c9247183 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:40.525106 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ff9b6d-1a78-4e44-a3c9-9fbfa9288e02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:40.525838 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ff9b6d-1a78-4e44-a3c9-9fbfa9288e02 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:40.526507 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ff9b6d-1a78-4e44-a3c9-9fbfa9288e02 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:40.526846 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ff9b6d-1a78-4e44-a3c9-9fbfa9288e02 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:40.533071 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:40.546722 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ff9b6d-1a78-4e44-a3c9-9fbfa9288e02 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:40.548363 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ff9b6d-1a78-4e44-a3c9-9fbfa9288e02 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:40.571794 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-a6ff9b6d-1a78-4e44-a3c9-9fbfa9288e02 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:17:40.571794 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:17:40.571794 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:17:40.571794 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:17:40.571794 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:17:40.571794 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:17:40.571794 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:17:40.571794 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:17:40.571794 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:17:40.571794 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:17:40.571794 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:17:40.571794 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:17:40.571794 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:17:40.571794 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:17:40.571794 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:17:40.571794 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:17:40.572815 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:17:40.572815 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:17:40.572815 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:17:40.572815 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:17:40.572815 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:17:40.572815 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:17:40.572815 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:17:40.572815 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Nov 02 07:17:40.572815 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:17:40.572815 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:17:40.572815 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:17:40.572815 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:17:40.572815 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:17:40.572815 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:17:40.572815 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:17:40.572815 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:17:40.572815 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:17:40.574964 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:17:40.574964 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:17:40.574964 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Nov 02 07:17:40.574964 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 328/661] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Nov 2 07:17:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:17:40.580002 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ff9b6d-1a78-4e44-a3c9-9fbfa9288e02 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:40.601684 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce773c6-1c79-4852-8910-63a874bc9c89 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:40.642000 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c3c819-4702-4831-969b-28e929a7329c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:40.642000 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c3c819-4702-4831-969b-28e929a7329c None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:40.643729 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c3c819-4702-4831-969b-28e929a7329c None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:40.643729 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c3c819-4702-4831-969b-28e929a7329c None admin] PATH_INFO: `/v3/services` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:40.643729 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c3c819-4702-4831-969b-28e929a7329c None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:17:40.643948 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c3c819-4702-4831-969b-28e929a7329c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:40.646695 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c3c819-4702-4831-969b-28e929a7329c None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:40.657079 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 334/662] 10.222.0.18 () {56 vars in 1122 bytes} [Wed Nov 2 07:17:40 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:17:40.658006 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c3c819-4702-4831-969b-28e929a7329c None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:40.712514 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ff9b6d-1a78-4e44-a3c9-9fbfa9288e02 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:40.748490 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f652b2-e023-41f0-90a1-2d8888073596 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:40.749528 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f652b2-e023-41f0-90a1-2d8888073596 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:40.750004 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f652b2-e023-41f0-90a1-2d8888073596 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:40.750525 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f652b2-e023-41f0-90a1-2d8888073596 None admin] PATH_INFO: `/v3/domains` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:40.751363 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f652b2-e023-41f0-90a1-2d8888073596 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:17:40.751893 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f652b2-e023-41f0-90a1-2d8888073596 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:40.753452 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f652b2-e023-41f0-90a1-2d8888073596 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:40.766403 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 329/663] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Nov 2 07:17:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:17:40.768296 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c3c819-4702-4831-969b-28e929a7329c None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:40.773237 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f652b2-e023-41f0-90a1-2d8888073596 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:40.777662 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:40.795292 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:40.809527 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fe3c64-16c3-4f7a-9c47-b325b8a227bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:40.810225 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fe3c64-16c3-4f7a-9c47-b325b8a227bb None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:40.810682 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fe3c64-16c3-4f7a-9c47-b325b8a227bb None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:40.810946 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fe3c64-16c3-4f7a-9c47-b325b8a227bb None admin] PATH_INFO: `/v3/services` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:40.811637 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fe3c64-16c3-4f7a-9c47-b325b8a227bb None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:17:40.811997 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fe3c64-16c3-4f7a-9c47-b325b8a227bb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:40.813680 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fe3c64-16c3-4f7a-9c47-b325b8a227bb None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:40.826395 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 335/664] 10.222.0.18 () {56 vars in 1122 bytes} [Wed Nov 2 07:17:40 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:17:40.904801 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f652b2-e023-41f0-90a1-2d8888073596 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:40.944127 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea80c40-0c16-4348-b39f-201d2e097a4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:40.944891 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea80c40-0c16-4348-b39f-201d2e097a4d None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:40.945169 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea80c40-0c16-4348-b39f-201d2e097a4d None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:40.945432 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea80c40-0c16-4348-b39f-201d2e097a4d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:40.953168 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea80c40-0c16-4348-b39f-201d2e097a4d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:40.954715 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea80c40-0c16-4348-b39f-201d2e097a4d None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:40.967402 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-3ea80c40-0c16-4348-b39f-201d2e097a4d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 07:17:40.967402 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:17:40.967402 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:17:40.967402 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:17:40.967402 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:17:40.967402 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:17:40.967402 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:17:40.967402 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:17:40.967402 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:17:40.967402 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:17:40.967402 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:17:40.967402 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:17:40.967402 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Nov 02 07:17:40.967402 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_project(project_id) Nov 02 07:17:40.967402 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Nov 02 07:17:40.968327 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Nov 02 07:17:40.968327 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:17:40.968327 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:17:40.968327 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:17:40.968327 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:17:40.968327 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:17:40.968327 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:17:40.968327 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:17:40.968327 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Nov 02 07:17:40.968327 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:17:40.968327 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:17:40.968327 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:17:40.968327 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:17:40.968327 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:17:40.968327 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:17:40.968327 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:17:40.968327 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:17:40.969212 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Nov 02 07:17:40.969212 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Nov 02 07:17:40.969212 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Nov 02 07:17:40.969212 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Nov 02 07:17:40.969212 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Nov 02 07:17:40.969212 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Nov 02 07:17:40.969212 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 07:17:40.969212 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Nov 02 07:17:40.969212 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 330/665] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Nov 2 07:17:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:17:40.976631 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fe3c64-16c3-4f7a-9c47-b325b8a227bb None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:41.094516 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fe3c64-16c3-4f7a-9c47-b325b8a227bb None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:41.130811 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461db553-ddcb-43f8-8fba-6c88033d83eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:41.131777 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461db553-ddcb-43f8-8fba-6c88033d83eb None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:41.132250 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461db553-ddcb-43f8-8fba-6c88033d83eb None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:41.132731 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461db553-ddcb-43f8-8fba-6c88033d83eb None admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:41.133614 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461db553-ddcb-43f8-8fba-6c88033d83eb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:17:41.134105 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461db553-ddcb-43f8-8fba-6c88033d83eb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:41.135792 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461db553-ddcb-43f8-8fba-6c88033d83eb None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:41.148558 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 336/666] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Nov 2 07:17:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:17:41.154596 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea80c40-0c16-4348-b39f-201d2e097a4d None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:41.221133 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:41.260480 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea80c40-0c16-4348-b39f-201d2e097a4d None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:41.295115 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa38f57-57d8-4392-8db1-301db6b77adb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:41.295765 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa38f57-57d8-4392-8db1-301db6b77adb None admin] REQUEST_METHOD: `PUT` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:41.296042 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa38f57-57d8-4392-8db1-301db6b77adb None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:41.296313 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa38f57-57d8-4392-8db1-301db6b77adb None admin] PATH_INFO: `/v3/projects/af04daa468b64160a6176dcab29502a7/users/ff4736f3d50746b0bf5d6a917b2ca0cf/roles/26344c06182042078907487e8d1a0d9f` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:41.298418 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa38f57-57d8-4392-8db1-301db6b77adb None admin] RBAC: Authorizing `identity:create_grant(project_id=af04daa468b64160a6176dcab29502a7, user_id=ff4736f3d50746b0bf5d6a917b2ca0cf, role_id=26344c06182042078907487e8d1a0d9f)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:41.299913 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa38f57-57d8-4392-8db1-301db6b77adb None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:41.316470 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 331/667] 10.222.0.18 () {58 vars in 1439 bytes} [Wed Nov 2 07:17:41 2022] PUT /identity/v3/projects/af04daa468b64160a6176dcab29502a7/users/ff4736f3d50746b0bf5d6a917b2ca0cf/roles/26344c06182042078907487e8d1a0d9f => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:17:42.392944 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc9f916-efb5-47e3-9ae0-efad903c326d None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:42.393493 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc9f916-efb5-47e3-9ae0-efad903c326d None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:42.393493 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc9f916-efb5-47e3-9ae0-efad903c326d None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:42.395259 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 337/668] 10.222.0.18 () {54 vars in 927 bytes} [Wed Nov 2 07:17:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:17:42.401438 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50eb034-4b93-45ab-acdf-31f36c258517 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:42.401954 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50eb034-4b93-45ab-acdf-31f36c258517 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:42.402062 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50eb034-4b93-45ab-acdf-31f36c258517 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:42.421927 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-e50eb034-4b93-45ab-acdf-31f36c258517 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:17:42.423151 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-e50eb034-4b93-45ab-acdf-31f36c258517 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:17:42.492242 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 332/669] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Nov 2 07:17:42 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 07:17:42.501148 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d281cd-5d77-4344-8a14-01516c7ff7c7 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:42.501434 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d281cd-5d77-4344-8a14-01516c7ff7c7 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:42.501709 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d281cd-5d77-4344-8a14-01516c7ff7c7 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:42.506649 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 338/670] 10.222.0.18 () {54 vars in 927 bytes} [Wed Nov 2 07:17:42 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:17:42.509239 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2983ffbe-55d0-4283-8ccd-cfb5fb811bd6 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:42.509529 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2983ffbe-55d0-4283-8ccd-cfb5fb811bd6 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:42.509808 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2983ffbe-55d0-4283-8ccd-cfb5fb811bd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:42.528120 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-2983ffbe-55d0-4283-8ccd-cfb5fb811bd6 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:17:42.529285 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-2983ffbe-55d0-4283-8ccd-cfb5fb811bd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:17:42.580382 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:42.597770 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 333/671] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Nov 2 07:17:42 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 07:17:42.772109 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d281cd-5d77-4344-8a14-01516c7ff7c7 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:42.821751 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:42.874291 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2983ffbe-55d0-4283-8ccd-cfb5fb811bd6 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:42.887387 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d281cd-5d77-4344-8a14-01516c7ff7c7 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:42.923862 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98581b10-1484-4808-ae4e-b5251027e40f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:42.924389 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98581b10-1484-4808-ae4e-b5251027e40f None admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:42.924668 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98581b10-1484-4808-ae4e-b5251027e40f None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:42.924939 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98581b10-1484-4808-ae4e-b5251027e40f None admin] PATH_INFO: `/v3/services` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:42.925606 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98581b10-1484-4808-ae4e-b5251027e40f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:42.927587 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98581b10-1484-4808-ae4e-b5251027e40f None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:42.950395 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 339/672] 10.222.0.18 () {60 vars in 1141 bytes} [Wed Nov 2 07:17:42 2022] POST /identity/v3/services => generated 251 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:17:42.999063 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2983ffbe-55d0-4283-8ccd-cfb5fb811bd6 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:43.049732 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256ea690-94dd-43fe-b494-9571f88e7364 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:43.050485 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256ea690-94dd-43fe-b494-9571f88e7364 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:43.050768 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256ea690-94dd-43fe-b494-9571f88e7364 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:43.051620 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256ea690-94dd-43fe-b494-9571f88e7364 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:43.052316 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256ea690-94dd-43fe-b494-9571f88e7364 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:43.053742 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256ea690-94dd-43fe-b494-9571f88e7364 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:43.064108 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-256ea690-94dd-43fe-b494-9571f88e7364 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Nov 02 07:17:43.064108 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:17:43.064108 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Nov 02 07:17:43.064108 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Nov 02 07:17:43.064108 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Nov 02 07:17:43.064108 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Nov 02 07:17:43.064108 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Nov 02 07:17:43.064108 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:17:43.064108 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Nov 02 07:17:43.064108 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Nov 02 07:17:43.064108 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 07:17:43.064108 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Nov 02 07:17:43.064108 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:17:43.064108 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:17:43.064108 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:17:43.064108 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:17:43.065141 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:17:43.065141 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:17:43.065141 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:17:43.065141 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:17:43.065141 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:17:43.065141 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:17:43.065141 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:17:43.065141 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Nov 02 07:17:43.065141 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_service(service_id) Nov 02 07:17:43.065141 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Nov 02 07:17:43.065141 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Nov 02 07:17:43.065141 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:17:43.065141 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:17:43.065141 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:17:43.065141 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:17:43.065141 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:17:43.065141 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:17:43.066015 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:17:43.066015 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Nov 02 07:17:43.066015 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:17:43.066015 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:17:43.066015 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:17:43.066015 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:17:43.066015 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:17:43.066015 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:17:43.066015 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:17:43.066015 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:17:43.066015 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Nov 02 07:17:43.066015 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:17:43.066015 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Nov 02 07:17:43.066015 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Nov 02 07:17:43.066015 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 334/673] 10.222.0.18 () {56 vars in 1132 bytes} [Wed Nov 2 07:17:42 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 07:17:43.072225 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98581b10-1484-4808-ae4e-b5251027e40f None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:43.156242 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4c7f78-e1cd-40ee-8714-de11217657b0 None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:43.156597 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4c7f78-e1cd-40ee-8714-de11217657b0 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:43.156795 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4c7f78-e1cd-40ee-8714-de11217657b0 None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:43.158136 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 335/674] 10.222.0.18 () {54 vars in 927 bytes} [Wed Nov 2 07:17:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:17:43.164610 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace5a3e0-bb8b-4bd9-829c-9c84629c97c3 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:43.164908 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace5a3e0-bb8b-4bd9-829c-9c84629c97c3 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:43.165152 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace5a3e0-bb8b-4bd9-829c-9c84629c97c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:43.184156 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-ace5a3e0-bb8b-4bd9-829c-9c84629c97c3 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:17:43.185255 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-ace5a3e0-bb8b-4bd9-829c-9c84629c97c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:17:43.196596 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98581b10-1484-4808-ae4e-b5251027e40f None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:43.205363 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:43.233195 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:43.241966 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11efa8b8-07d4-4623-9a82-caed0ed37574 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:43.242615 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11efa8b8-07d4-4623-9a82-caed0ed37574 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:43.243011 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11efa8b8-07d4-4623-9a82-caed0ed37574 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:43.243428 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11efa8b8-07d4-4623-9a82-caed0ed37574 None admin] PATH_INFO: `/v3/services` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:43.244060 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11efa8b8-07d4-4623-9a82-caed0ed37574 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:17:43.244362 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11efa8b8-07d4-4623-9a82-caed0ed37574 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:43.246265 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11efa8b8-07d4-4623-9a82-caed0ed37574 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:43.252762 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 340/675] 10.222.0.18 () {56 vars in 1126 bytes} [Wed Nov 2 07:17:43 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:17:43.258625 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11efa8b8-07d4-4623-9a82-caed0ed37574 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:43.268282 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 336/676] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Nov 2 07:17:43 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Nov 02 07:17:43.390226 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11efa8b8-07d4-4623-9a82-caed0ed37574 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:43.430618 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58770bf-5746-4d0f-a23f-b314e0b9d570 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:43.431944 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58770bf-5746-4d0f-a23f-b314e0b9d570 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:43.432371 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58770bf-5746-4d0f-a23f-b314e0b9d570 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:43.432784 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58770bf-5746-4d0f-a23f-b314e0b9d570 None admin] PATH_INFO: `/v3/services` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:43.433648 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58770bf-5746-4d0f-a23f-b314e0b9d570 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:17:43.434128 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58770bf-5746-4d0f-a23f-b314e0b9d570 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:43.435763 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58770bf-5746-4d0f-a23f-b314e0b9d570 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:43.442571 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 341/677] 10.222.0.18 () {56 vars in 1126 bytes} [Wed Nov 2 07:17:43 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:17:43.450855 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace5a3e0-bb8b-4bd9-829c-9c84629c97c3 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:43.523768 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f592955b-688e-4d79-91b1-5ba5296fc667 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:43.524044 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f592955b-688e-4d79-91b1-5ba5296fc667 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:43.524320 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f592955b-688e-4d79-91b1-5ba5296fc667 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:43.525642 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 342/678] 10.222.0.18 () {54 vars in 927 bytes} [Wed Nov 2 07:17:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Nov 02 07:17:43.531234 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b1956d-8362-4364-a5a1-4e9c2a6abcd8 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:43.531516 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b1956d-8362-4364-a5a1-4e9c2a6abcd8 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:43.531791 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b1956d-8362-4364-a5a1-4e9c2a6abcd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:43.550617 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-a2b1956d-8362-4364-a5a1-4e9c2a6abcd8 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:17:43.552452 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-a2b1956d-8362-4364-a5a1-4e9c2a6abcd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:17:43.562691 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace5a3e0-bb8b-4bd9-829c-9c84629c97c3 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:43.588614 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:43.600865 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661caf03-dc6c-49cb-a984-29f8e84b7f33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:43.601472 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661caf03-dc6c-49cb-a984-29f8e84b7f33 None admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:43.601746 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661caf03-dc6c-49cb-a984-29f8e84b7f33 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:43.602008 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661caf03-dc6c-49cb-a984-29f8e84b7f33 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:43.602949 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661caf03-dc6c-49cb-a984-29f8e84b7f33 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:43.604314 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661caf03-dc6c-49cb-a984-29f8e84b7f33 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:43.630626 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 337/679] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Nov 2 07:17:43 2022] POST /identity/v3/endpoints => generated 347 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:17:43.632753 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661caf03-dc6c-49cb-a984-29f8e84b7f33 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:43.639518 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 343/680] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Nov 2 07:17:43 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 07:17:43.673446 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:43.755871 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661caf03-dc6c-49cb-a984-29f8e84b7f33 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:43.793818 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2aea3e2-13ee-4a10-a009-b7a5dfe07b21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:43.794528 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2aea3e2-13ee-4a10-a009-b7a5dfe07b21 None admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:43.795156 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2aea3e2-13ee-4a10-a009-b7a5dfe07b21 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:43.795427 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2aea3e2-13ee-4a10-a009-b7a5dfe07b21 None admin] PATH_INFO: `/v3/services` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:43.796094 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2aea3e2-13ee-4a10-a009-b7a5dfe07b21 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:43.797555 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2aea3e2-13ee-4a10-a009-b7a5dfe07b21 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:43.815733 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 338/681] 10.222.0.18 () {60 vars in 1141 bytes} [Wed Nov 2 07:17:43 2022] POST /identity/v3/services => generated 241 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:17:43.933932 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b1956d-8362-4364-a5a1-4e9c2a6abcd8 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:44.046890 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b1956d-8362-4364-a5a1-4e9c2a6abcd8 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:44.099924 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44278533-ebde-4bc0-894f-6a03bbdd0504 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:44.100805 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44278533-ebde-4bc0-894f-6a03bbdd0504 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:44.101281 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44278533-ebde-4bc0-894f-6a03bbdd0504 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:44.101681 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44278533-ebde-4bc0-894f-6a03bbdd0504 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:44.116887 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44278533-ebde-4bc0-894f-6a03bbdd0504 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:17:44.122129 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-44278533-ebde-4bc0-894f-6a03bbdd0504 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Nov 02 07:17:44.122129 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:17:44.122129 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:17:44.122129 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:17:44.122129 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:17:44.122129 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:17:44.122129 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:17:44.122129 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:17:44.122129 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:17:44.122129 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:17:44.122129 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:17:44.122129 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:17:44.122129 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:17:44.122129 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:17:44.122129 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:17:44.122129 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:17:44.123309 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:17:44.123309 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:17:44.123309 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:17:44.123309 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:17:44.123309 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:17:44.123309 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:17:44.123309 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:17:44.123309 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:17:44.123309 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:17:44.123309 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:17:44.123309 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:17:44.123309 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:17:44.123309 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:17:44.123309 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:17:44.123309 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:17:44.123309 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:17:44.124245 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:17:44.124245 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:17:44.124245 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:17:44.124245 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:17:44.124245 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:17:44.124245 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:17:44.124245 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:17:44.124245 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:17:44.124245 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:17:44.124245 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:17:44.124245 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Nov 02 07:17:44.124245 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:17:44.126458 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 344/682] 10.222.0.18 () {56 vars in 1120 bytes} [Wed Nov 2 07:17:43 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:17:44.139803 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2aea3e2-13ee-4a10-a009-b7a5dfe07b21 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:44.277010 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:44.287342 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2aea3e2-13ee-4a10-a009-b7a5dfe07b21 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:44.325652 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8643cd40-938b-4827-87f2-f9935006192f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:44.326180 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8643cd40-938b-4827-87f2-f9935006192f None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:44.326492 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8643cd40-938b-4827-87f2-f9935006192f None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:44.326769 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8643cd40-938b-4827-87f2-f9935006192f None admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:44.327436 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8643cd40-938b-4827-87f2-f9935006192f None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:17:44.327738 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8643cd40-938b-4827-87f2-f9935006192f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:44.329092 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8643cd40-938b-4827-87f2-f9935006192f None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:44.342499 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 339/683] 10.222.0.18 () {56 vars in 1115 bytes} [Wed Nov 2 07:17:44 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:17:44.344391 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44278533-ebde-4bc0-894f-6a03bbdd0504 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:44.381445 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:44.473963 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44278533-ebde-4bc0-894f-6a03bbdd0504 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:44.516992 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430ca8c3-ae4e-4762-b5d0-8addf32aa80f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:44.518609 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430ca8c3-ae4e-4762-b5d0-8addf32aa80f None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:44.518887 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430ca8c3-ae4e-4762-b5d0-8addf32aa80f None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:44.519155 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430ca8c3-ae4e-4762-b5d0-8addf32aa80f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:44.538502 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430ca8c3-ae4e-4762-b5d0-8addf32aa80f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:44.539976 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430ca8c3-ae4e-4762-b5d0-8addf32aa80f None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:44.571375 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-430ca8c3-ae4e-4762-b5d0-8addf32aa80f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:17:44.571375 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:17:44.571375 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:17:44.571375 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:17:44.571375 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:17:44.571375 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:17:44.571375 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:17:44.571375 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:17:44.571375 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:17:44.571375 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:17:44.571375 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:17:44.571375 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:17:44.571375 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:17:44.571375 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:17:44.571375 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:17:44.571375 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:17:44.572382 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:17:44.572382 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:17:44.572382 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:17:44.572382 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:17:44.572382 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:17:44.572382 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:17:44.572382 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:17:44.572382 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:17:44.572382 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:17:44.572382 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:17:44.572382 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:17:44.572382 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:17:44.572382 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:17:44.572382 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:17:44.572382 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:17:44.572382 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:17:44.572382 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:17:44.573247 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:17:44.573247 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:17:44.573247 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:17:44.573247 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 345/684] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Nov 2 07:17:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:17:44.579106 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8643cd40-938b-4827-87f2-f9935006192f None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:44.736415 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8643cd40-938b-4827-87f2-f9935006192f None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:44.771447 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ac5acd-2f00-4a63-abd3-663411dd13fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:44.772413 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ac5acd-2f00-4a63-abd3-663411dd13fd None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:44.772886 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ac5acd-2f00-4a63-abd3-663411dd13fd None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:44.773351 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ac5acd-2f00-4a63-abd3-663411dd13fd None admin] PATH_INFO: `/v3/domains` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:44.774182 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ac5acd-2f00-4a63-abd3-663411dd13fd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:17:44.774711 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ac5acd-2f00-4a63-abd3-663411dd13fd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:44.776275 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ac5acd-2f00-4a63-abd3-663411dd13fd None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:44.789031 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 340/685] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Nov 2 07:17:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:17:44.794620 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430ca8c3-ae4e-4762-b5d0-8addf32aa80f None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:44.862627 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:44.903172 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430ca8c3-ae4e-4762-b5d0-8addf32aa80f None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:44.940195 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4307228-029b-417b-973c-3600017f73f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:44.940827 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4307228-029b-417b-973c-3600017f73f1 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:44.941107 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4307228-029b-417b-973c-3600017f73f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:44.941382 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4307228-029b-417b-973c-3600017f73f1 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:44.949672 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4307228-029b-417b-973c-3600017f73f1 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:44.951162 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4307228-029b-417b-973c-3600017f73f1 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:44.963195 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-b4307228-029b-417b-973c-3600017f73f1 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Nov 02 07:17:44.963195 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:17:44.963195 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:17:44.963195 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:17:44.963195 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:17:44.963195 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:17:44.963195 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:17:44.963195 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:17:44.963195 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:17:44.963195 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:17:44.963195 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:17:44.963195 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:17:44.963195 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Nov 02 07:17:44.963195 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_user(user_id) Nov 02 07:17:44.963195 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Nov 02 07:17:44.963195 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Nov 02 07:17:44.964123 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:17:44.964123 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:17:44.964123 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Nov 02 07:17:44.964123 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:17:44.964123 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Nov 02 07:17:44.964123 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:17:44.964123 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:17:44.964123 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:17:44.964123 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:17:44.964123 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:17:44.964123 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:17:44.964123 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:17:44.964123 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:17:44.964123 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:17:44.964123 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:17:44.964123 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:17:44.964123 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:17:44.964123 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:17:44.965074 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:17:44.965074 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:17:44.965074 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Nov 02 07:17:44.965074 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Nov 02 07:17:44.965074 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Nov 02 07:17:44.965074 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Nov 02 07:17:44.965074 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Nov 02 07:17:44.965074 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Nov 02 07:17:44.965074 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Nov 02 07:17:44.965074 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:17:44.965074 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 346/686] 10.222.0.18 () {56 vars in 1117 bytes} [Wed Nov 2 07:17:44 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:17:44.970101 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ac5acd-2f00-4a63-abd3-663411dd13fd None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:44.998750 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:45.079870 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ac5acd-2f00-4a63-abd3-663411dd13fd None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:45.115182 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b310a4-5699-46aa-a073-43c5e69612e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:45.116160 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b310a4-5699-46aa-a073-43c5e69612e4 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:45.116675 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b310a4-5699-46aa-a073-43c5e69612e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:45.117155 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b310a4-5699-46aa-a073-43c5e69612e4 None admin] PATH_INFO: `/v3/users` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:45.118064 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b310a4-5699-46aa-a073-43c5e69612e4 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:17:45.118580 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b310a4-5699-46aa-a073-43c5e69612e4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:45.120191 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b310a4-5699-46aa-a073-43c5e69612e4 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:45.139259 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 341/687] 10.222.0.18 () {56 vars in 1149 bytes} [Wed Nov 2 07:17:44 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:17:45.146405 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4307228-029b-417b-973c-3600017f73f1 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:45.253215 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4307228-029b-417b-973c-3600017f73f1 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:45.282408 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:45.291261 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f41b257-9b56-4a1e-a07a-dcde75e19801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:45.291883 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f41b257-9b56-4a1e-a07a-dcde75e19801 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:45.292164 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f41b257-9b56-4a1e-a07a-dcde75e19801 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:45.292434 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f41b257-9b56-4a1e-a07a-dcde75e19801 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:45.365919 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f41b257-9b56-4a1e-a07a-dcde75e19801 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:45.367339 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f41b257-9b56-4a1e-a07a-dcde75e19801 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:45.407908 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711d418e-1398-4e0b-9680-77139d655c41 None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:45.408229 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711d418e-1398-4e0b-9680-77139d655c41 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:45.408460 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711d418e-1398-4e0b-9680-77139d655c41 None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:45.410134 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 342/688] 10.222.0.18 () {54 vars in 927 bytes} [Wed Nov 2 07:17:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:17:45.418939 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbe1057-5f07-4624-a2d1-cc790b85ae81 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:45.419474 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbe1057-5f07-4624-a2d1-cc790b85ae81 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:45.419945 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbe1057-5f07-4624-a2d1-cc790b85ae81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:45.443034 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-7fbe1057-5f07-4624-a2d1-cc790b85ae81 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:17:45.444546 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-7fbe1057-5f07-4624-a2d1-cc790b85ae81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:17:45.481481 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 347/689] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Nov 2 07:17:45 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Nov 02 07:17:45.483600 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f41b257-9b56-4a1e-a07a-dcde75e19801 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:45.514542 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:45.527551 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:45.534791 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 343/690] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Nov 2 07:17:45 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Nov 02 07:17:45.597076 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f41b257-9b56-4a1e-a07a-dcde75e19801 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:45.634445 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247fdf64-08ce-49f6-a63b-9c06f5c8d6e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:45.635014 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247fdf64-08ce-49f6-a63b-9c06f5c8d6e0 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:45.635293 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247fdf64-08ce-49f6-a63b-9c06f5c8d6e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:45.635564 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247fdf64-08ce-49f6-a63b-9c06f5c8d6e0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:45.646351 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247fdf64-08ce-49f6-a63b-9c06f5c8d6e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:45.649693 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247fdf64-08ce-49f6-a63b-9c06f5c8d6e0 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:45.664086 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-247fdf64-08ce-49f6-a63b-9c06f5c8d6e0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:17:45.664086 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:17:45.664086 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:17:45.664086 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:17:45.664086 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:17:45.664086 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:17:45.664086 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:17:45.664086 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:17:45.664086 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:17:45.664086 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:17:45.664086 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:17:45.664086 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:17:45.664086 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:17:45.664086 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:17:45.664086 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:17:45.664086 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:17:45.665041 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:17:45.665041 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:17:45.665041 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:17:45.665041 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:17:45.665041 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:17:45.665041 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:17:45.665041 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:17:45.665041 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:17:45.665041 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:17:45.665041 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:17:45.665041 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:17:45.665041 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:17:45.665041 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:17:45.665041 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:17:45.665041 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:17:45.665041 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:17:45.665041 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:17:45.666164 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:17:45.666164 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:17:45.666164 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:17:45.666164 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 348/691] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Nov 2 07:17:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 07:17:45.672449 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbe1057-5f07-4624-a2d1-cc790b85ae81 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:45.775869 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbe1057-5f07-4624-a2d1-cc790b85ae81 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:45.810456 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc419863-116c-4094-bd12-7f0404844020 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:45.811060 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc419863-116c-4094-bd12-7f0404844020 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:45.811337 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc419863-116c-4094-bd12-7f0404844020 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:45.811604 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc419863-116c-4094-bd12-7f0404844020 None admin] PATH_INFO: `/v3/domains` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:45.812234 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc419863-116c-4094-bd12-7f0404844020 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:17:45.812529 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc419863-116c-4094-bd12-7f0404844020 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:45.813951 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc419863-116c-4094-bd12-7f0404844020 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:45.825820 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 344/692] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Nov 2 07:17:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:17:45.831515 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247fdf64-08ce-49f6-a63b-9c06f5c8d6e0 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:45.841708 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc419863-116c-4094-bd12-7f0404844020 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:45.904222 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:45.937189 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:45.964249 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc419863-116c-4094-bd12-7f0404844020 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:45.971183 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247fdf64-08ce-49f6-a63b-9c06f5c8d6e0 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:46.004324 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62032a4a-867e-4a0a-b709-3f59404b33ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:46.005342 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62032a4a-867e-4a0a-b709-3f59404b33ba None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:46.005840 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62032a4a-867e-4a0a-b709-3f59404b33ba None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:46.006295 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62032a4a-867e-4a0a-b709-3f59404b33ba None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:46.006764 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3610b5-1f7f-424b-9275-d9e29fc80631 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:46.007287 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62032a4a-867e-4a0a-b709-3f59404b33ba None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:46.007505 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3610b5-1f7f-424b-9275-d9e29fc80631 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:46.007864 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3610b5-1f7f-424b-9275-d9e29fc80631 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:46.008219 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3610b5-1f7f-424b-9275-d9e29fc80631 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:46.009471 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62032a4a-867e-4a0a-b709-3f59404b33ba None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:46.015418 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3610b5-1f7f-424b-9275-d9e29fc80631 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:46.016933 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3610b5-1f7f-424b-9275-d9e29fc80631 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:46.020367 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-62032a4a-867e-4a0a-b709-3f59404b33ba None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Nov 02 07:17:46.020367 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:17:46.020367 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Nov 02 07:17:46.020367 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Nov 02 07:17:46.020367 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Nov 02 07:17:46.020367 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Nov 02 07:17:46.020367 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Nov 02 07:17:46.020367 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:17:46.020367 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Nov 02 07:17:46.020367 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Nov 02 07:17:46.020367 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 07:17:46.020367 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Nov 02 07:17:46.020367 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:17:46.020367 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:17:46.020367 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:17:46.020367 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:17:46.020367 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:17:46.021559 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:17:46.021559 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:17:46.021559 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:17:46.021559 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:17:46.021559 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:17:46.021559 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:17:46.021559 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Nov 02 07:17:46.021559 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_service(service_id) Nov 02 07:17:46.021559 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Nov 02 07:17:46.021559 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Nov 02 07:17:46.021559 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:17:46.021559 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:17:46.021559 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:17:46.021559 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:17:46.021559 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:17:46.021559 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:17:46.022468 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:17:46.022468 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Nov 02 07:17:46.022468 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:17:46.022468 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:17:46.022468 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:17:46.022468 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:17:46.022468 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:17:46.022468 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:17:46.022468 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:17:46.022468 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:17:46.022468 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Nov 02 07:17:46.022468 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:17:46.022468 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Nov 02 07:17:46.022468 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Nov 02 07:17:46.025405 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 345/693] 10.222.0.18 () {56 vars in 1114 bytes} [Wed Nov 2 07:17:45 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 07:17:46.034643 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-1a3610b5-1f7f-424b-9275-d9e29fc80631 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 07:17:46.034643 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:17:46.034643 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:17:46.034643 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:17:46.034643 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:17:46.034643 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:17:46.034643 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:17:46.034643 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:17:46.034643 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:17:46.034643 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:17:46.034643 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:17:46.034643 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:17:46.034643 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Nov 02 07:17:46.034643 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_project(project_id) Nov 02 07:17:46.034643 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Nov 02 07:17:46.035699 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Nov 02 07:17:46.035699 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:17:46.035699 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:17:46.035699 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:17:46.035699 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:17:46.035699 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:17:46.035699 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:17:46.035699 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:17:46.035699 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:17:46.035699 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:17:46.035699 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:17:46.035699 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:17:46.035699 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:17:46.035699 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:17:46.035699 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:17:46.035699 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:17:46.035699 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:17:46.036567 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Nov 02 07:17:46.036567 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Nov 02 07:17:46.036567 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Nov 02 07:17:46.036567 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Nov 02 07:17:46.036567 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Nov 02 07:17:46.036567 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Nov 02 07:17:46.036567 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 07:17:46.036567 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:17:46.036990 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62032a4a-867e-4a0a-b709-3f59404b33ba None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:46.037081 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 349/694] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Nov 2 07:17:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:17:46.037509 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3610b5-1f7f-424b-9275-d9e29fc80631 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:46.147415 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3610b5-1f7f-424b-9275-d9e29fc80631 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:46.147776 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62032a4a-867e-4a0a-b709-3f59404b33ba None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:46.174041 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:46.184992 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1d2c03-6f6e-439d-9312-1007f51915ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:46.185595 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1d2c03-6f6e-439d-9312-1007f51915ab None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:46.185873 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1d2c03-6f6e-439d-9312-1007f51915ab None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:46.185975 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be66abb-ec38-4bcc-a9bb-71de480460ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:46.186148 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1d2c03-6f6e-439d-9312-1007f51915ab None admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:46.186539 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be66abb-ec38-4bcc-a9bb-71de480460ad None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:46.186821 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be66abb-ec38-4bcc-a9bb-71de480460ad None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:46.186916 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1d2c03-6f6e-439d-9312-1007f51915ab None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:17:46.187092 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be66abb-ec38-4bcc-a9bb-71de480460ad None admin] PATH_INFO: `/v3/services` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:46.187188 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1d2c03-6f6e-439d-9312-1007f51915ab None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:46.187712 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be66abb-ec38-4bcc-a9bb-71de480460ad None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:17:46.188012 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be66abb-ec38-4bcc-a9bb-71de480460ad None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:46.188561 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1d2c03-6f6e-439d-9312-1007f51915ab None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:46.189358 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be66abb-ec38-4bcc-a9bb-71de480460ad None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:46.195078 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 346/695] 10.222.0.18 () {56 vars in 1114 bytes} [Wed Nov 2 07:17:46 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:17:46.196750 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f394bc-370c-4ea4-8a40-0b3a770e945f None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:46.197038 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f394bc-370c-4ea4-8a40-0b3a770e945f None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:46.197308 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f394bc-370c-4ea4-8a40-0b3a770e945f None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:46.197806 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:46.199126 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 347/696] 10.222.0.18 () {54 vars in 927 bytes} [Wed Nov 2 07:17:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:17:46.200783 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f394bc-370c-4ea4-8a40-0b3a770e945f None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:46.206659 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 350/697] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Nov 2 07:17:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:17:46.208317 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0befacc-176e-42ff-8e75-7d7f942d4f4e None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:46.208849 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0befacc-176e-42ff-8e75-7d7f942d4f4e None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:46.209322 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0befacc-176e-42ff-8e75-7d7f942d4f4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:46.231719 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-a0befacc-176e-42ff-8e75-7d7f942d4f4e None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:17:46.232916 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-a0befacc-176e-42ff-8e75-7d7f942d4f4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:17:46.298574 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 351/698] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Nov 2 07:17:46 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 07:17:46.299398 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0befacc-176e-42ff-8e75-7d7f942d4f4e None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:46.309379 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f394bc-370c-4ea4-8a40-0b3a770e945f None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:46.344544 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f11bc56-3b2b-43dd-a409-5f9e88b1b73d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:46.345155 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f11bc56-3b2b-43dd-a409-5f9e88b1b73d None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:46.345435 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f11bc56-3b2b-43dd-a409-5f9e88b1b73d None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:46.345705 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f11bc56-3b2b-43dd-a409-5f9e88b1b73d None admin] PATH_INFO: `/v3/services` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:46.346330 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f11bc56-3b2b-43dd-a409-5f9e88b1b73d None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:17:46.346662 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f11bc56-3b2b-43dd-a409-5f9e88b1b73d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:46.348530 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f11bc56-3b2b-43dd-a409-5f9e88b1b73d None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:46.355939 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 348/699] 10.222.0.18 () {56 vars in 1114 bytes} [Wed Nov 2 07:17:46 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:17:46.358427 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f095e7d-d119-4592-af11-b077c4cb1841 None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:46.358877 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f095e7d-d119-4592-af11-b077c4cb1841 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:46.359305 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f095e7d-d119-4592-af11-b077c4cb1841 None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:46.361508 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 349/700] 10.222.0.18 () {54 vars in 927 bytes} [Wed Nov 2 07:17:46 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:17:46.362413 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f095e7d-d119-4592-af11-b077c4cb1841 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:46.411356 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0befacc-176e-42ff-8e75-7d7f942d4f4e None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:46.424481 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:46.440268 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:46.446946 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb385134-f94d-4a4a-ac10-43ae60c064e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:46.447551 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb385134-f94d-4a4a-ac10-43ae60c064e2 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:46.447827 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb385134-f94d-4a4a-ac10-43ae60c064e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:46.448093 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb385134-f94d-4a4a-ac10-43ae60c064e2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:46.448807 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb385134-f94d-4a4a-ac10-43ae60c064e2 None admin] RBAC: Adding query filter params (role.id=26344c06182042078907487e8d1a0d9f, scope.project.id=af04daa468b64160a6176dcab29502a7, user.id=ff4736f3d50746b0bf5d6a917b2ca0cf) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:17:46.449109 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb385134-f94d-4a4a-ac10-43ae60c064e2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:46.450538 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb385134-f94d-4a4a-ac10-43ae60c064e2 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:46.456643 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 352/701] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Nov 2 07:17:46 2022] GET /identity/v3/role_assignments?user.id=ff4736f3d50746b0bf5d6a917b2ca0cf&scope.project.id=af04daa468b64160a6176dcab29502a7&role.id=26344c06182042078907487e8d1a0d9f => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:17:46.458635 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8fe69c-78cc-41e6-8b2f-3c863914a15b None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:46.458908 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8fe69c-78cc-41e6-8b2f-3c863914a15b None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:46.459173 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8fe69c-78cc-41e6-8b2f-3c863914a15b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:46.479203 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-5c8fe69c-78cc-41e6-8b2f-3c863914a15b None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:17:46.480674 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-5c8fe69c-78cc-41e6-8b2f-3c863914a15b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:17:46.493120 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f095e7d-d119-4592-af11-b077c4cb1841 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:46.531363 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af9c551-d7d9-4c94-9672-7ccb9a157d3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:46.532220 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af9c551-d7d9-4c94-9672-7ccb9a157d3d None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:46.532596 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af9c551-d7d9-4c94-9672-7ccb9a157d3d None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:46.532963 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af9c551-d7d9-4c94-9672-7ccb9a157d3d None admin] PATH_INFO: `/v3/endpoints` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:46.533805 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af9c551-d7d9-4c94-9672-7ccb9a157d3d None admin] RBAC: Adding query filter params (interface=public, service_id=ca8fe6ba559340fa93b541afb817b6b1, region_id=RegionOne) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:17:46.534211 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af9c551-d7d9-4c94-9672-7ccb9a157d3d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:46.535846 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af9c551-d7d9-4c94-9672-7ccb9a157d3d None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:46.546151 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 350/702] 10.222.0.18 () {56 vars in 1251 bytes} [Wed Nov 2 07:17:46 2022] GET /identity/v3/endpoints?service_id=ca8fe6ba559340fa93b541afb817b6b1&interface=public®ion_id=RegionOne => generated 194 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:17:46.547703 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af9c551-d7d9-4c94-9672-7ccb9a157d3d None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:46.578389 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 353/703] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Nov 2 07:17:46 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 07:17:46.578389 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8fe69c-78cc-41e6-8b2f-3c863914a15b None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:46.668984 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af9c551-d7d9-4c94-9672-7ccb9a157d3d None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:46.689499 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:46.711230 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee116f11-5bfb-4628-be10-2b4496241544 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:46.711834 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee116f11-5bfb-4628-be10-2b4496241544 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:46.712113 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee116f11-5bfb-4628-be10-2b4496241544 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:46.712385 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee116f11-5bfb-4628-be10-2b4496241544 None admin] PATH_INFO: `/v3/services` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:46.712983 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee116f11-5bfb-4628-be10-2b4496241544 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:46.713557 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8fe69c-78cc-41e6-8b2f-3c863914a15b None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:46.714345 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee116f11-5bfb-4628-be10-2b4496241544 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:46.722613 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 351/704] 10.222.0.18 () {56 vars in 1087 bytes} [Wed Nov 2 07:17:46 2022] GET /identity/v3/services => generated 1497 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 07:17:46.743479 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:46.756535 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7115a0d9-6909-43e2-8a49-3ee44bd1ebc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:46.757193 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7115a0d9-6909-43e2-8a49-3ee44bd1ebc8 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:46.757475 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7115a0d9-6909-43e2-8a49-3ee44bd1ebc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:46.757749 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7115a0d9-6909-43e2-8a49-3ee44bd1ebc8 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:46.758342 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7115a0d9-6909-43e2-8a49-3ee44bd1ebc8 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:46.759768 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7115a0d9-6909-43e2-8a49-3ee44bd1ebc8 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:46.770247 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-7115a0d9-6909-43e2-8a49-3ee44bd1ebc8 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Nov 02 07:17:46.770247 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:17:46.770247 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Nov 02 07:17:46.770247 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Nov 02 07:17:46.770247 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Nov 02 07:17:46.770247 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Nov 02 07:17:46.770247 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Nov 02 07:17:46.770247 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:17:46.770247 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Nov 02 07:17:46.770247 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:17:46.770247 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 07:17:46.770247 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:17:46.770247 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:17:46.770247 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:17:46.770247 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:17:46.770247 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:17:46.770247 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:17:46.771444 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:17:46.771444 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:17:46.771444 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:17:46.771444 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:17:46.771444 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:17:46.771444 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:17:46.771444 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Nov 02 07:17:46.771444 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_service(service_id) Nov 02 07:17:46.771444 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Nov 02 07:17:46.771444 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Nov 02 07:17:46.771444 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:17:46.771444 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:17:46.771444 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:17:46.771444 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:17:46.771444 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:17:46.771444 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:17:46.772274 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:17:46.772274 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:17:46.772274 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:17:46.772274 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:17:46.772274 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:17:46.772274 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:17:46.772274 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:17:46.772274 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:17:46.772274 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:17:46.772274 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:17:46.772274 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Nov 02 07:17:46.772274 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:17:46.772274 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Nov 02 07:17:46.772274 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:17:46.772274 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 354/705] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Nov 2 07:17:46 2022] GET /identity/v3/services/compute => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:17:46.777724 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee116f11-5bfb-4628-be10-2b4496241544 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:46.865767 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7115a0d9-6909-43e2-8a49-3ee44bd1ebc8 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:46.901536 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee116f11-5bfb-4628-be10-2b4496241544 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:46.954473 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921ec1f1-113a-4c70-b8f3-42d13fa1c592 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:46.954473 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921ec1f1-113a-4c70-b8f3-42d13fa1c592 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:46.954473 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921ec1f1-113a-4c70-b8f3-42d13fa1c592 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:46.954473 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921ec1f1-113a-4c70-b8f3-42d13fa1c592 None admin] PATH_INFO: `/v3/services` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:46.954874 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921ec1f1-113a-4c70-b8f3-42d13fa1c592 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:17:46.954874 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921ec1f1-113a-4c70-b8f3-42d13fa1c592 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:46.954874 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921ec1f1-113a-4c70-b8f3-42d13fa1c592 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:46.960802 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 352/706] 10.222.0.18 () {56 vars in 1112 bytes} [Wed Nov 2 07:17:46 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:17:46.965901 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921ec1f1-113a-4c70-b8f3-42d13fa1c592 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:46.994174 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7115a0d9-6909-43e2-8a49-3ee44bd1ebc8 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:47.029721 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cccd53a-2622-4235-a01f-e9a1f56601b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:47.030436 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cccd53a-2622-4235-a01f-e9a1f56601b3 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:47.030729 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cccd53a-2622-4235-a01f-e9a1f56601b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:47.031001 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cccd53a-2622-4235-a01f-e9a1f56601b3 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:47.031497 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:47.031611 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cccd53a-2622-4235-a01f-e9a1f56601b3 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:47.032950 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cccd53a-2622-4235-a01f-e9a1f56601b3 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:47.043235 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-8cccd53a-2622-4235-a01f-e9a1f56601b3 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Nov 02 07:17:47.043235 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:17:47.043235 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Nov 02 07:17:47.043235 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Nov 02 07:17:47.043235 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Nov 02 07:17:47.043235 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Nov 02 07:17:47.043235 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Nov 02 07:17:47.043235 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:17:47.043235 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Nov 02 07:17:47.043235 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:17:47.043235 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 07:17:47.043235 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:17:47.043235 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:17:47.043235 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:17:47.043235 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:17:47.043235 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:17:47.043235 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:17:47.044472 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:17:47.044472 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:17:47.044472 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:17:47.044472 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:17:47.044472 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:17:47.044472 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:17:47.044472 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Nov 02 07:17:47.044472 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_service(service_id) Nov 02 07:17:47.044472 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Nov 02 07:17:47.044472 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Nov 02 07:17:47.044472 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:17:47.044472 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:17:47.044472 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:17:47.044472 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:17:47.044472 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:17:47.044472 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:17:47.045467 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:17:47.045467 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:17:47.045467 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:17:47.045467 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:17:47.045467 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:17:47.045467 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:17:47.045467 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:17:47.045467 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:17:47.045467 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:17:47.045467 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:17:47.045467 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Nov 02 07:17:47.045467 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:17:47.045467 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Nov 02 07:17:47.045467 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:17:47.045467 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 355/707] 10.222.0.18 () {56 vars in 1126 bytes} [Wed Nov 2 07:17:46 2022] GET /identity/v3/services/object-store => generated 93 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 07:17:47.051318 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cccd53a-2622-4235-a01f-e9a1f56601b3 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:47.090972 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921ec1f1-113a-4c70-b8f3-42d13fa1c592 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:47.124670 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:47.147258 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d12ea3-5740-46a1-8492-ac6723a824a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:47.147873 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d12ea3-5740-46a1-8492-ac6723a824a9 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:47.148150 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d12ea3-5740-46a1-8492-ac6723a824a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:47.148417 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d12ea3-5740-46a1-8492-ac6723a824a9 None admin] PATH_INFO: `/v3/services` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:47.149067 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d12ea3-5740-46a1-8492-ac6723a824a9 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:17:47.149366 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d12ea3-5740-46a1-8492-ac6723a824a9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:47.150797 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d12ea3-5740-46a1-8492-ac6723a824a9 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:47.156180 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 353/708] 10.222.0.18 () {56 vars in 1112 bytes} [Wed Nov 2 07:17:46 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:17:47.193583 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cccd53a-2622-4235-a01f-e9a1f56601b3 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:47.233656 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f786cc7c-7ba3-49e9-9a37-5bf9212bebd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:47.234208 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f786cc7c-7ba3-49e9-9a37-5bf9212bebd2 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:47.234519 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f786cc7c-7ba3-49e9-9a37-5bf9212bebd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:47.234790 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f786cc7c-7ba3-49e9-9a37-5bf9212bebd2 None admin] PATH_INFO: `/v3/services` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:47.235449 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f786cc7c-7ba3-49e9-9a37-5bf9212bebd2 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:17:47.235816 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f786cc7c-7ba3-49e9-9a37-5bf9212bebd2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:47.237244 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f786cc7c-7ba3-49e9-9a37-5bf9212bebd2 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:47.244232 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 356/709] 10.222.0.18 () {56 vars in 1122 bytes} [Wed Nov 2 07:17:47 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:17:47.251577 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d12ea3-5740-46a1-8492-ac6723a824a9 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:47.375507 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d12ea3-5740-46a1-8492-ac6723a824a9 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:47.387534 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:47.415198 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54d3189-99d2-4c1e-b9b8-5e8ccb64e878 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:47.416200 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54d3189-99d2-4c1e-b9b8-5e8ccb64e878 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:47.416674 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54d3189-99d2-4c1e-b9b8-5e8ccb64e878 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:47.417136 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54d3189-99d2-4c1e-b9b8-5e8ccb64e878 None admin] PATH_INFO: `/v3/services` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:47.417947 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54d3189-99d2-4c1e-b9b8-5e8ccb64e878 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:17:47.418487 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54d3189-99d2-4c1e-b9b8-5e8ccb64e878 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:47.420021 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54d3189-99d2-4c1e-b9b8-5e8ccb64e878 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:47.425709 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 354/710] 10.222.0.18 () {56 vars in 1122 bytes} [Wed Nov 2 07:17:47 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:17:47.434450 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f786cc7c-7ba3-49e9-9a37-5bf9212bebd2 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:47.545116 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f786cc7c-7ba3-49e9-9a37-5bf9212bebd2 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:47.562707 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:47.581355 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393a723c-019d-49c8-b282-421da36b53ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:47.581964 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393a723c-019d-49c8-b282-421da36b53ce None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:47.582280 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393a723c-019d-49c8-b282-421da36b53ce None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:47.583131 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393a723c-019d-49c8-b282-421da36b53ce None admin] PATH_INFO: `/v3/endpoints` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:47.583839 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393a723c-019d-49c8-b282-421da36b53ce None admin] RBAC: Adding query filter params (interface=public, service_id=508c32ed86cf420483d2aeffbb07c27a, region_id=RegionOne) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:17:47.584146 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393a723c-019d-49c8-b282-421da36b53ce None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:47.585510 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393a723c-019d-49c8-b282-421da36b53ce None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:47.591064 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 357/711] 10.222.0.18 () {56 vars in 1251 bytes} [Wed Nov 2 07:17:47 2022] GET /identity/v3/endpoints?service_id=508c32ed86cf420483d2aeffbb07c27a&interface=public®ion_id=RegionOne => generated 194 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:17:47.596592 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54d3189-99d2-4c1e-b9b8-5e8ccb64e878 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:47.704769 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54d3189-99d2-4c1e-b9b8-5e8ccb64e878 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:47.741835 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818bbeba-7624-4037-83c8-6ffb8974cb8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:47.742462 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818bbeba-7624-4037-83c8-6ffb8974cb8e None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:47.742745 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818bbeba-7624-4037-83c8-6ffb8974cb8e None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:47.743014 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818bbeba-7624-4037-83c8-6ffb8974cb8e None admin] PATH_INFO: `/v3/services` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:47.743612 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818bbeba-7624-4037-83c8-6ffb8974cb8e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:47.744978 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818bbeba-7624-4037-83c8-6ffb8974cb8e None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:47.751377 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 355/712] 10.222.0.18 () {56 vars in 1087 bytes} [Wed Nov 2 07:17:47 2022] GET /identity/v3/services => generated 1497 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:17:48.794605 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f05fd1-668f-45b3-9071-b0828a8832ce None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:48.795165 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f05fd1-668f-45b3-9071-b0828a8832ce None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:48.795165 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f05fd1-668f-45b3-9071-b0828a8832ce None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:48.796730 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 358/713] 10.222.0.18 () {54 vars in 927 bytes} [Wed Nov 2 07:17:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:17:48.803092 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2ddad4-5686-4f85-9b0f-956cf4c88834 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:48.803502 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2ddad4-5686-4f85-9b0f-956cf4c88834 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:48.803668 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2ddad4-5686-4f85-9b0f-956cf4c88834 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:48.821933 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-ee2ddad4-5686-4f85-9b0f-956cf4c88834 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:17:48.823151 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-ee2ddad4-5686-4f85-9b0f-956cf4c88834 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:17:48.856439 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:48.899380 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 356/714] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Nov 2 07:17:48 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 07:17:48.971182 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d70b5dd-3221-454c-a624-3c7bcaa98f02 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:48.971467 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d70b5dd-3221-454c-a624-3c7bcaa98f02 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:48.971745 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d70b5dd-3221-454c-a624-3c7bcaa98f02 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:48.973437 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 359/715] 10.222.0.18 () {54 vars in 927 bytes} [Wed Nov 2 07:17:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:17:48.979052 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa23fb8-1cec-4928-a534-06295d275e89 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:48.979523 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa23fb8-1cec-4928-a534-06295d275e89 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:48.979949 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa23fb8-1cec-4928-a534-06295d275e89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:48.999070 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-4aa23fb8-1cec-4928-a534-06295d275e89 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:17:49.000447 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-4aa23fb8-1cec-4928-a534-06295d275e89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:17:49.069817 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 357/716] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Nov 2 07:17:48 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 07:17:49.173791 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d70b5dd-3221-454c-a624-3c7bcaa98f02 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:49.280928 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d70b5dd-3221-454c-a624-3c7bcaa98f02 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:49.316226 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f5a3e6-e656-4a1f-b771-e49173981185 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:49.316889 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f5a3e6-e656-4a1f-b771-e49173981185 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:49.317271 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f5a3e6-e656-4a1f-b771-e49173981185 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:49.317552 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f5a3e6-e656-4a1f-b771-e49173981185 None admin] PATH_INFO: `/v3/services/image` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:49.318137 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f5a3e6-e656-4a1f-b771-e49173981185 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:49.319535 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f5a3e6-e656-4a1f-b771-e49173981185 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:49.328494 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-e6f5a3e6-e656-4a1f-b771-e49173981185 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Nov 02 07:17:49.328494 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:17:49.328494 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Nov 02 07:17:49.328494 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Nov 02 07:17:49.328494 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Nov 02 07:17:49.328494 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Nov 02 07:17:49.328494 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Nov 02 07:17:49.328494 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:17:49.328494 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Nov 02 07:17:49.328494 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:17:49.328494 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 07:17:49.328494 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:17:49.328494 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:17:49.328494 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:17:49.328494 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:17:49.328494 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:17:49.328494 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:17:49.329470 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:17:49.329470 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:17:49.329470 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:17:49.329470 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:17:49.329470 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:17:49.329470 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:17:49.329470 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Nov 02 07:17:49.329470 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_service(service_id) Nov 02 07:17:49.329470 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Nov 02 07:17:49.329470 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Nov 02 07:17:49.329470 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:17:49.329470 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:17:49.329470 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:17:49.329470 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:17:49.329470 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:17:49.329470 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:17:49.330294 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:17:49.330294 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:17:49.330294 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:17:49.330294 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:17:49.330294 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:17:49.330294 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:17:49.330294 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:17:49.330294 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:17:49.330294 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:17:49.330294 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:17:49.330294 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Nov 02 07:17:49.330294 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:17:49.330294 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Nov 02 07:17:49.330294 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:17:49.330294 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 360/717] 10.222.0.18 () {56 vars in 1105 bytes} [Wed Nov 2 07:17:49 2022] GET /identity/v3/services/image => generated 86 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:17:49.335760 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa23fb8-1cec-4928-a534-06295d275e89 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:49.346895 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f5a3e6-e656-4a1f-b771-e49173981185 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:49.408202 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:49.469319 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:49.479417 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa23fb8-1cec-4928-a534-06295d275e89 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:49.484593 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f5a3e6-e656-4a1f-b771-e49173981185 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:49.516761 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556a4226-97a6-40dc-a1f3-68254861fba6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:49.517473 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556a4226-97a6-40dc-a1f3-68254861fba6 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:49.517813 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556a4226-97a6-40dc-a1f3-68254861fba6 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:49.518087 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556a4226-97a6-40dc-a1f3-68254861fba6 None admin] PATH_INFO: `/v3/services` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:49.518763 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556a4226-97a6-40dc-a1f3-68254861fba6 None admin] RBAC: Adding query filter params (name=image) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:17:49.519067 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556a4226-97a6-40dc-a1f3-68254861fba6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:49.521008 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa70b5c9-114e-4987-9a3c-33c684db568b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:49.521143 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556a4226-97a6-40dc-a1f3-68254861fba6 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:49.521641 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa70b5c9-114e-4987-9a3c-33c684db568b None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:49.521920 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa70b5c9-114e-4987-9a3c-33c684db568b None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:49.522393 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa70b5c9-114e-4987-9a3c-33c684db568b None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:49.523300 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa70b5c9-114e-4987-9a3c-33c684db568b None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:49.524646 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa70b5c9-114e-4987-9a3c-33c684db568b None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:49.530264 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 358/718] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Nov 2 07:17:49 2022] GET /identity/v3/services?name=image => generated 122 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:17:49.530264 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14cdcc9-481f-44a7-b473-1cb9269830be None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:49.530264 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14cdcc9-481f-44a7-b473-1cb9269830be None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:49.530264 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14cdcc9-481f-44a7-b473-1cb9269830be None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:49.530264 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 359/719] 10.222.0.18 () {54 vars in 927 bytes} [Wed Nov 2 07:17:49 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:17:49.533548 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14cdcc9-481f-44a7-b473-1cb9269830be None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:49.538954 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-fa70b5c9-114e-4987-9a3c-33c684db568b None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Nov 02 07:17:49.538954 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:17:49.538954 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Nov 02 07:17:49.538954 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Nov 02 07:17:49.538954 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Nov 02 07:17:49.538954 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Nov 02 07:17:49.538954 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Nov 02 07:17:49.538954 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:17:49.538954 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Nov 02 07:17:49.538954 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:17:49.538954 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 07:17:49.538954 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:17:49.538954 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:17:49.538954 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:17:49.538954 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:17:49.538954 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:17:49.538954 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:17:49.539949 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:17:49.539949 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:17:49.539949 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:17:49.539949 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:17:49.539949 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:17:49.539949 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:17:49.539949 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Nov 02 07:17:49.539949 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_service(service_id) Nov 02 07:17:49.539949 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Nov 02 07:17:49.539949 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Nov 02 07:17:49.539949 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:17:49.539949 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:17:49.539949 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:17:49.539949 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:17:49.539949 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:17:49.539949 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:17:49.541396 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:17:49.541396 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:17:49.541396 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:17:49.541396 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:17:49.541396 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:17:49.541396 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:17:49.541396 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:17:49.541396 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:17:49.541396 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:17:49.541396 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:17:49.541396 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Nov 02 07:17:49.541396 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:17:49.541396 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Nov 02 07:17:49.541396 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:17:49.541396 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 361/720] 10.222.0.18 () {56 vars in 1114 bytes} [Wed Nov 2 07:17:49 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 07:17:49.542204 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90651642-cc7e-4795-a983-bc34f5502796 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:49.542204 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90651642-cc7e-4795-a983-bc34f5502796 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:49.542204 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90651642-cc7e-4795-a983-bc34f5502796 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:49.560124 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-90651642-cc7e-4795-a983-bc34f5502796 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:17:49.561363 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-90651642-cc7e-4795-a983-bc34f5502796 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:17:49.635692 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 362/721] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Nov 2 07:17:49 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 07:17:49.636340 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90651642-cc7e-4795-a983-bc34f5502796 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:49.664592 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14cdcc9-481f-44a7-b473-1cb9269830be None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:49.685984 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:49.714212 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbacc95-8289-4bb9-9ef5-25f7b634779f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:49.715405 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbacc95-8289-4bb9-9ef5-25f7b634779f None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:49.716443 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbacc95-8289-4bb9-9ef5-25f7b634779f None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:49.716927 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbacc95-8289-4bb9-9ef5-25f7b634779f None admin] PATH_INFO: `/v3/services` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:49.718037 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbacc95-8289-4bb9-9ef5-25f7b634779f None admin] RBAC: Adding query filter params (type=image) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:17:49.718810 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbacc95-8289-4bb9-9ef5-25f7b634779f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:49.721327 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbacc95-8289-4bb9-9ef5-25f7b634779f None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:49.734826 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 360/722] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Nov 2 07:17:49 2022] GET /identity/v3/services?type=image => generated 122 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:17:49.759534 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90651642-cc7e-4795-a983-bc34f5502796 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:49.799778 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788102e1-bae5-4c02-9aac-18d41f7cc762 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:49.800460 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788102e1-bae5-4c02-9aac-18d41f7cc762 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:49.800722 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788102e1-bae5-4c02-9aac-18d41f7cc762 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:49.801545 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788102e1-bae5-4c02-9aac-18d41f7cc762 None admin] PATH_INFO: `/v3/services` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:49.802256 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788102e1-bae5-4c02-9aac-18d41f7cc762 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:17:49.802591 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788102e1-bae5-4c02-9aac-18d41f7cc762 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:49.804036 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788102e1-bae5-4c02-9aac-18d41f7cc762 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:49.810061 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 363/723] 10.222.0.18 () {56 vars in 1114 bytes} [Wed Nov 2 07:17:49 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:17:49.815516 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbacc95-8289-4bb9-9ef5-25f7b634779f None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:49.844959 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:49.909390 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788102e1-bae5-4c02-9aac-18d41f7cc762 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:49.937832 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbacc95-8289-4bb9-9ef5-25f7b634779f None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:49.979394 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54a09cc-3462-4604-acec-4582b7752464 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:49.980236 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54a09cc-3462-4604-acec-4582b7752464 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:49.980519 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54a09cc-3462-4604-acec-4582b7752464 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:49.980789 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54a09cc-3462-4604-acec-4582b7752464 None admin] PATH_INFO: `/v3/services` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:49.981420 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54a09cc-3462-4604-acec-4582b7752464 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:17:49.981722 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54a09cc-3462-4604-acec-4582b7752464 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:49.983146 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54a09cc-3462-4604-acec-4582b7752464 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:49.988103 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 361/724] 10.222.0.18 () {56 vars in 1114 bytes} [Wed Nov 2 07:17:49 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:17:49.989850 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5aa6b17-390b-4525-8f68-c1c9edc7527a None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:49.990154 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5aa6b17-390b-4525-8f68-c1c9edc7527a None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:49.990442 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5aa6b17-390b-4525-8f68-c1c9edc7527a None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:49.991939 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 362/725] 10.222.0.18 () {54 vars in 927 bytes} [Wed Nov 2 07:17:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:17:49.994022 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5aa6b17-390b-4525-8f68-c1c9edc7527a None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:50.037824 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788102e1-bae5-4c02-9aac-18d41f7cc762 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:50.039485 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:50.074518 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64c19f9-44ed-4215-9b9d-60a3ee739f08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:50.075128 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64c19f9-44ed-4215-9b9d-60a3ee739f08 None admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:50.075407 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64c19f9-44ed-4215-9b9d-60a3ee739f08 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:50.075676 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64c19f9-44ed-4215-9b9d-60a3ee739f08 None admin] PATH_INFO: `/v3/services` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:50.076334 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64c19f9-44ed-4215-9b9d-60a3ee739f08 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:50.077774 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64c19f9-44ed-4215-9b9d-60a3ee739f08 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:50.092440 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 364/726] 10.222.0.18 () {60 vars in 1141 bytes} [Wed Nov 2 07:17:49 2022] POST /identity/v3/services => generated 242 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 07:17:50.093727 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7d1c13-197f-4fb7-97ed-4a0d5aa5129e None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:50.094012 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7d1c13-197f-4fb7-97ed-4a0d5aa5129e None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:50.094281 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7d1c13-197f-4fb7-97ed-4a0d5aa5129e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:50.099169 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:50.115675 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-1b7d1c13-197f-4fb7-97ed-4a0d5aa5129e None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:17:50.116964 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-1b7d1c13-197f-4fb7-97ed-4a0d5aa5129e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:17:50.134847 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5aa6b17-390b-4525-8f68-c1c9edc7527a None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:50.191847 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ca2c99-2737-4312-bfba-9d2bf9bf8e71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:50.193316 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ca2c99-2737-4312-bfba-9d2bf9bf8e71 None admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:50.194265 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ca2c99-2737-4312-bfba-9d2bf9bf8e71 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:50.195037 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ca2c99-2737-4312-bfba-9d2bf9bf8e71 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:50.195919 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ca2c99-2737-4312-bfba-9d2bf9bf8e71 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:50.197783 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ca2c99-2737-4312-bfba-9d2bf9bf8e71 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:50.199511 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 365/727] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Nov 2 07:17:50 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 07:17:50.228252 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 363/728] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Nov 2 07:17:49 2022] POST /identity/v3/endpoints => generated 346 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 07:17:50.469849 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7d1c13-197f-4fb7-97ed-4a0d5aa5129e None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:50.566897 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:50.576826 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7d1c13-197f-4fb7-97ed-4a0d5aa5129e None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:50.620984 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9287ad-3b28-4586-9361-ecdd713fd787 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:50.622023 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9287ad-3b28-4586-9361-ecdd713fd787 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:50.622546 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9287ad-3b28-4586-9361-ecdd713fd787 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:50.623022 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9287ad-3b28-4586-9361-ecdd713fd787 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:50.623800 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9287ad-3b28-4586-9361-ecdd713fd787 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:50.625366 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9287ad-3b28-4586-9361-ecdd713fd787 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:50.635336 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-ff9287ad-3b28-4586-9361-ecdd713fd787 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Nov 02 07:17:50.635336 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:17:50.635336 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Nov 02 07:17:50.635336 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Nov 02 07:17:50.635336 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Nov 02 07:17:50.635336 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Nov 02 07:17:50.635336 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Nov 02 07:17:50.635336 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:17:50.635336 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Nov 02 07:17:50.635336 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:17:50.635336 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 07:17:50.635336 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:17:50.635336 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:17:50.635336 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:17:50.635336 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:17:50.635336 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:17:50.635336 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:17:50.636338 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:17:50.636338 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:17:50.636338 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:17:50.636338 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:17:50.636338 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:17:50.636338 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:17:50.636338 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Nov 02 07:17:50.636338 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_service(service_id) Nov 02 07:17:50.636338 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Nov 02 07:17:50.636338 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Nov 02 07:17:50.636338 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:17:50.636338 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:17:50.636338 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:17:50.636338 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:17:50.636338 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:17:50.636338 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:17:50.637193 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:17:50.637193 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:17:50.637193 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:17:50.637193 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:17:50.637193 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:17:50.637193 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:17:50.637193 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:17:50.637193 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:17:50.637193 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:17:50.637193 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:17:50.637193 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Nov 02 07:17:50.637193 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:17:50.637193 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Nov 02 07:17:50.637193 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:17:50.639898 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 366/729] 10.222.0.18 () {56 vars in 1126 bytes} [Wed Nov 2 07:17:50 2022] GET /identity/v3/services/object-store => generated 93 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:17:50.647659 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ca2c99-2737-4312-bfba-9d2bf9bf8e71 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:50.758483 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ca2c99-2737-4312-bfba-9d2bf9bf8e71 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:50.793289 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c43e8c9-e6e4-4a42-a553-adebb37e0ec6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:50.794244 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c43e8c9-e6e4-4a42-a553-adebb37e0ec6 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:50.794774 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c43e8c9-e6e4-4a42-a553-adebb37e0ec6 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:50.795241 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c43e8c9-e6e4-4a42-a553-adebb37e0ec6 None admin] PATH_INFO: `/v3/services` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:50.796059 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c43e8c9-e6e4-4a42-a553-adebb37e0ec6 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:17:50.796575 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c43e8c9-e6e4-4a42-a553-adebb37e0ec6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:50.798131 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c43e8c9-e6e4-4a42-a553-adebb37e0ec6 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:50.803679 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 364/730] 10.222.0.18 () {56 vars in 1122 bytes} [Wed Nov 2 07:17:50 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:17:50.808966 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9287ad-3b28-4586-9361-ecdd713fd787 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:50.921071 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9287ad-3b28-4586-9361-ecdd713fd787 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:50.955442 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bbb8ed-1f09-48a7-8818-ba8c960810a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:50.956055 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bbb8ed-1f09-48a7-8818-ba8c960810a5 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:50.956339 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bbb8ed-1f09-48a7-8818-ba8c960810a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:50.956614 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bbb8ed-1f09-48a7-8818-ba8c960810a5 None admin] PATH_INFO: `/v3/services` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:50.957769 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bbb8ed-1f09-48a7-8818-ba8c960810a5 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:17:50.958070 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bbb8ed-1f09-48a7-8818-ba8c960810a5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:50.959455 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bbb8ed-1f09-48a7-8818-ba8c960810a5 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:50.964753 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 367/731] 10.222.0.18 () {56 vars in 1122 bytes} [Wed Nov 2 07:17:50 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:17:50.970385 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c43e8c9-e6e4-4a42-a553-adebb37e0ec6 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:50.975287 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:51.084675 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c43e8c9-e6e4-4a42-a553-adebb37e0ec6 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:51.121125 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea6be12-76cf-4d6a-ab22-0579700793f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:51.121736 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea6be12-76cf-4d6a-ab22-0579700793f0 None admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:51.122016 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea6be12-76cf-4d6a-ab22-0579700793f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:51.122287 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea6be12-76cf-4d6a-ab22-0579700793f0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:51.122987 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea6be12-76cf-4d6a-ab22-0579700793f0 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:51.124338 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea6be12-76cf-4d6a-ab22-0579700793f0 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:51.149679 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 365/732] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Nov 2 07:17:50 2022] POST /identity/v3/endpoints => generated 349 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:17:52.042309 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0014f386-7fd4-41dd-91ec-e7d7f76eb945 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:52.042869 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0014f386-7fd4-41dd-91ec-e7d7f76eb945 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:52.042869 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0014f386-7fd4-41dd-91ec-e7d7f76eb945 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:52.044599 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 368/733] 10.222.0.18 () {54 vars in 927 bytes} [Wed Nov 2 07:17:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:17:52.050669 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f867c25c-6638-4c44-88c2-80ca5b9171f4 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:52.051786 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f867c25c-6638-4c44-88c2-80ca5b9171f4 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:52.051786 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f867c25c-6638-4c44-88c2-80ca5b9171f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:52.070859 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-f867c25c-6638-4c44-88c2-80ca5b9171f4 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:17:52.072089 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-f867c25c-6638-4c44-88c2-80ca5b9171f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:17:52.131389 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:52.154468 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 366/734] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Nov 2 07:17:52 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 07:17:52.385993 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d5d92d-36ca-4042-b6bc-d59bcbd8320e None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:52.386691 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d5d92d-36ca-4042-b6bc-d59bcbd8320e None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:52.387183 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d5d92d-36ca-4042-b6bc-d59bcbd8320e None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:52.389010 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 369/735] 10.222.0.18 () {54 vars in 927 bytes} [Wed Nov 2 07:17:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:17:52.395449 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c18070f-a5a6-49c8-8462-9c4a2bc3cfda None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:52.396017 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c18070f-a5a6-49c8-8462-9c4a2bc3cfda None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:52.396512 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c18070f-a5a6-49c8-8462-9c4a2bc3cfda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:52.415478 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-6c18070f-a5a6-49c8-8462-9c4a2bc3cfda None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:17:52.416988 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-6c18070f-a5a6-49c8-8462-9c4a2bc3cfda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:17:52.428428 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d5d92d-36ca-4042-b6bc-d59bcbd8320e None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:52.451998 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:52.502511 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 367/736] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Nov 2 07:17:52 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 07:17:52.503861 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a291f7-3fbc-469f-9553-f37d46d3c379 None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:52.504290 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a291f7-3fbc-469f-9553-f37d46d3c379 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:52.504569 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a291f7-3fbc-469f-9553-f37d46d3c379 None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:52.506125 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 368/737] 10.222.0.18 () {54 vars in 927 bytes} [Wed Nov 2 07:17:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:17:52.514389 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03021da-0832-4cc5-b08f-c19655243689 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:52.514389 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03021da-0832-4cc5-b08f-c19655243689 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:52.514389 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03021da-0832-4cc5-b08f-c19655243689 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:52.531355 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-a03021da-0832-4cc5-b08f-c19655243689 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:17:52.532534 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-a03021da-0832-4cc5-b08f-c19655243689 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:17:52.553580 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d5d92d-36ca-4042-b6bc-d59bcbd8320e None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:52.588071 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a2232b-a789-42c1-93aa-40fb2f584e84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:52.588676 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a2232b-a789-42c1-93aa-40fb2f584e84 None admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:52.588959 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a2232b-a789-42c1-93aa-40fb2f584e84 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:52.589237 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a2232b-a789-42c1-93aa-40fb2f584e84 None admin] PATH_INFO: `/v3/services` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:52.589897 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a2232b-a789-42c1-93aa-40fb2f584e84 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:52.591286 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a2232b-a789-42c1-93aa-40fb2f584e84 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:52.609140 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 370/738] 10.222.0.18 () {60 vars in 1141 bytes} [Wed Nov 2 07:17:52 2022] POST /identity/v3/services => generated 242 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 07:17:52.614529 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 369/739] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Nov 2 07:17:52 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Nov 02 07:17:52.800319 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a2232b-a789-42c1-93aa-40fb2f584e84 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:52.881408 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:52.882290 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03021da-0832-4cc5-b08f-c19655243689 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:52.931794 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a2232b-a789-42c1-93aa-40fb2f584e84 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:52.938595 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:52.970230 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2ad4b3-2747-43e1-b41c-316250482158 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:52.971238 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2ad4b3-2747-43e1-b41c-316250482158 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:52.971667 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2ad4b3-2747-43e1-b41c-316250482158 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:52.972122 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2ad4b3-2747-43e1-b41c-316250482158 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:52.973650 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2ad4b3-2747-43e1-b41c-316250482158 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:52.975331 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2ad4b3-2747-43e1-b41c-316250482158 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:52.985841 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-4a2ad4b3-2747-43e1-b41c-316250482158 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Nov 02 07:17:52.985841 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:17:52.985841 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Nov 02 07:17:52.985841 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Nov 02 07:17:52.985841 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Nov 02 07:17:52.985841 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Nov 02 07:17:52.985841 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Nov 02 07:17:52.985841 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:17:52.985841 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Nov 02 07:17:52.985841 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:17:52.985841 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 07:17:52.985841 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:17:52.985841 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:17:52.985841 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:17:52.985841 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:17:52.985841 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:17:52.985841 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:17:52.987026 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:17:52.987026 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:17:52.987026 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:17:52.987026 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:17:52.987026 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:17:52.987026 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:17:52.987026 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Nov 02 07:17:52.987026 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_service(service_id) Nov 02 07:17:52.987026 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Nov 02 07:17:52.987026 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Nov 02 07:17:52.987026 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:17:52.987026 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:17:52.987026 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:17:52.987026 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:17:52.987026 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:17:52.987026 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:17:52.988092 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:17:52.988092 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:17:52.988092 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:17:52.988092 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:17:52.988092 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:17:52.988092 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:17:52.988092 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:17:52.988092 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:17:52.988092 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:17:52.988092 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:17:52.988092 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Nov 02 07:17:52.988092 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:17:52.988092 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Nov 02 07:17:52.988092 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:17:52.990695 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 371/740] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Nov 2 07:17:52 2022] GET /identity/v3/services/compute => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:17:52.996609 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2ad4b3-2747-43e1-b41c-316250482158 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:53.009532 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03021da-0832-4cc5-b08f-c19655243689 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:53.055894 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc3ad55-7e86-4989-a96d-f6745a652935 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:53.056533 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc3ad55-7e86-4989-a96d-f6745a652935 None admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:53.056781 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc3ad55-7e86-4989-a96d-f6745a652935 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:53.057056 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc3ad55-7e86-4989-a96d-f6745a652935 None admin] PATH_INFO: `/v3/projects` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:53.058157 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc3ad55-7e86-4989-a96d-f6745a652935 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:53.059934 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc3ad55-7e86-4989-a96d-f6745a652935 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:53.063363 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: WARNING keystone.server.flask.common [None req-4bc3ad55-7e86-4989-a96d-f6745a652935 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Nov 02 07:17:53.086955 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.sql.core [None req-4bc3ad55-7e86-4989-a96d-f6745a652935 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Nov 02 07:17:53.086955 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Nov 02 07:17:53.086955 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [parameters: {'id': 'bfecf63fb6dd4082bd446279bc2b63e5', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Nov 02 07:17:53.086955 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81501) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Nov 02 07:17:53.096032 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-4bc3ad55-7e86-4989-a96d-f6745a652935 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id bfecf63fb6dd4082bd446279bc2b63e5.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id bfecf63fb6dd4082bd446279bc2b63e5. Nov 02 07:17:53.096032 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:17:53.096032 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Nov 02 07:17:53.096032 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application self.dialect.do_execute( Nov 02 07:17:53.096032 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Nov 02 07:17:53.096032 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Nov 02 07:17:53.096032 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Nov 02 07:17:53.096032 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application result = self._query(query) Nov 02 07:17:53.096032 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Nov 02 07:17:53.096032 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application conn.query(q) Nov 02 07:17:53.096032 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Nov 02 07:17:53.096032 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Nov 02 07:17:53.096032 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Nov 02 07:17:53.096931 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application result.read() Nov 02 07:17:53.096931 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Nov 02 07:17:53.096931 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Nov 02 07:17:53.096931 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Nov 02 07:17:53.096931 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application packet.raise_for_error() Nov 02 07:17:53.096931 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Nov 02 07:17:53.096931 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Nov 02 07:17:53.096931 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Nov 02 07:17:53.096931 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Nov 02 07:17:53.096931 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Nov 02 07:17:53.096931 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Nov 02 07:17:53.096931 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Nov 02 07:17:53.096931 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Nov 02 07:17:53.096931 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:17:53.096931 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Nov 02 07:17:53.096931 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return method(*args, **kwargs) Nov 02 07:17:53.096931 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Nov 02 07:17:53.096931 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return project_ref.to_dict() Nov 02 07:17:53.098743 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Nov 02 07:17:53.098743 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application next(self.gen) Nov 02 07:17:53.098743 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1068, in _transaction_scope Nov 02 07:17:53.098743 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application yield resource Nov 02 07:17:53.098743 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Nov 02 07:17:53.098743 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application next(self.gen) Nov 02 07:17:53.098743 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 670, in _session Nov 02 07:17:53.098743 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application self.session.rollback() Nov 02 07:17:53.098743 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Nov 02 07:17:53.098743 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application self.force_reraise() Nov 02 07:17:53.098743 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Nov 02 07:17:53.098743 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise self.value Nov 02 07:17:53.098743 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 667, in _session Nov 02 07:17:53.098743 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Nov 02 07:17:53.098743 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 695, in _end_session_transaction Nov 02 07:17:53.098743 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application session.commit() Nov 02 07:17:53.098743 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Nov 02 07:17:53.099775 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Nov 02 07:17:53.099775 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Nov 02 07:17:53.099775 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application self._prepare_impl() Nov 02 07:17:53.099775 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Nov 02 07:17:53.099775 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application self.session.flush() Nov 02 07:17:53.099775 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Nov 02 07:17:53.099775 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application self._flush(objects) Nov 02 07:17:53.099775 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Nov 02 07:17:53.099775 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Nov 02 07:17:53.099775 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Nov 02 07:17:53.099775 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application compat.raise_( Nov 02 07:17:53.099775 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Nov 02 07:17:53.099775 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception Nov 02 07:17:53.099775 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Nov 02 07:17:53.099775 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application flush_context.execute() Nov 02 07:17:53.099775 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Nov 02 07:17:53.099775 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rec.execute(self) Nov 02 07:17:53.102488 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Nov 02 07:17:53.102488 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Nov 02 07:17:53.102488 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Nov 02 07:17:53.102488 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application _emit_insert_statements( Nov 02 07:17:53.102488 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Nov 02 07:17:53.102488 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application c = connection._execute_20( Nov 02 07:17:53.102488 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Nov 02 07:17:53.102488 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Nov 02 07:17:53.102488 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Nov 02 07:17:53.102488 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Nov 02 07:17:53.102488 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Nov 02 07:17:53.102488 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application ret = self._execute_context( Nov 02 07:17:53.102488 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Nov 02 07:17:53.102488 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Nov 02 07:17:53.102488 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Nov 02 07:17:53.105100 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Nov 02 07:17:53.105100 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Nov 02 07:17:53.105100 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception Nov 02 07:17:53.105100 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Nov 02 07:17:53.105100 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application self.dialect.do_execute( Nov 02 07:17:53.105100 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Nov 02 07:17:53.105100 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Nov 02 07:17:53.105100 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Nov 02 07:17:53.105100 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application result = self._query(query) Nov 02 07:17:53.105100 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Nov 02 07:17:53.105100 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application conn.query(q) Nov 02 07:17:53.105100 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Nov 02 07:17:53.105100 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Nov 02 07:17:53.105100 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Nov 02 07:17:53.105100 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application result.read() Nov 02 07:17:53.105100 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Nov 02 07:17:53.105100 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Nov 02 07:17:53.106525 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Nov 02 07:17:53.106525 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application packet.raise_for_error() Nov 02 07:17:53.106525 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Nov 02 07:17:53.106525 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Nov 02 07:17:53.106525 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Nov 02 07:17:53.106525 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Nov 02 07:17:53.106525 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Nov 02 07:17:53.106525 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Nov 02 07:17:53.106525 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [parameters: {'id': 'bfecf63fb6dd4082bd446279bc2b63e5', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Nov 02 07:17:53.106525 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Nov 02 07:17:53.106525 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Nov 02 07:17:53.106525 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 07:17:53.106525 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Nov 02 07:17:53.106525 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:17:53.106525 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Nov 02 07:17:53.108301 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Nov 02 07:17:53.108301 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Nov 02 07:17:53.108301 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Nov 02 07:17:53.108301 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Nov 02 07:17:53.108301 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Nov 02 07:17:53.108301 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 07:17:53.108301 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Nov 02 07:17:53.108301 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:17:53.108301 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:17:53.108301 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:17:53.108301 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:17:53.108301 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:17:53.108301 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:17:53.108301 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:17:53.108301 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:17:53.108301 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:17:53.108301 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:17:53.111064 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:17:53.111064 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Nov 02 07:17:53.111064 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Nov 02 07:17:53.111064 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:17:53.111064 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:17:53.111064 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Nov 02 07:17:53.111064 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.Conflict( Nov 02 07:17:53.111064 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id bfecf63fb6dd4082bd446279bc2b63e5. Nov 02 07:17:53.111064 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Nov 02 07:17:53.111064 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 370/741] 10.222.0.18 () {60 vars in 1140 bytes} [Wed Nov 2 07:17:52 2022] POST /identity/v3/projects => generated 262 bytes in 216 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Nov 02 07:17:53.111064 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc3ad55-7e86-4989-a96d-f6745a652935 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:53.126632 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2ad4b3-2747-43e1-b41c-316250482158 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:53.174068 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92531383-e825-40a1-8e62-386050157309 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:53.174828 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92531383-e825-40a1-8e62-386050157309 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:53.175161 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92531383-e825-40a1-8e62-386050157309 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:53.175502 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92531383-e825-40a1-8e62-386050157309 None admin] PATH_INFO: `/v3/services` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:53.176867 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92531383-e825-40a1-8e62-386050157309 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:17:53.177235 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92531383-e825-40a1-8e62-386050157309 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:53.179325 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92531383-e825-40a1-8e62-386050157309 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:53.181979 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:53.193091 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 372/742] 10.222.0.18 () {56 vars in 1112 bytes} [Wed Nov 2 07:17:52 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:17:53.195304 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92531383-e825-40a1-8e62-386050157309 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:53.244130 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc3ad55-7e86-4989-a96d-f6745a652935 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:53.246566 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:53.299090 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7121f631-0714-43bd-a338-1a2990c64e8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:53.300029 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7121f631-0714-43bd-a338-1a2990c64e8e None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:53.300515 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7121f631-0714-43bd-a338-1a2990c64e8e None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:53.301662 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7121f631-0714-43bd-a338-1a2990c64e8e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:53.316264 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92531383-e825-40a1-8e62-386050157309 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:53.320372 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7121f631-0714-43bd-a338-1a2990c64e8e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:53.322369 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7121f631-0714-43bd-a338-1a2990c64e8e None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:53.339361 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-7121f631-0714-43bd-a338-1a2990c64e8e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 07:17:53.339361 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:17:53.339361 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:17:53.339361 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:17:53.339361 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:17:53.339361 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:17:53.339361 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:17:53.339361 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:17:53.339361 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:17:53.339361 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:17:53.339361 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:17:53.339361 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:17:53.339361 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Nov 02 07:17:53.339361 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_project(project_id) Nov 02 07:17:53.339361 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Nov 02 07:17:53.340376 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Nov 02 07:17:53.340376 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:17:53.340376 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:17:53.340376 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:17:53.340376 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:17:53.340376 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:17:53.340376 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:17:53.340376 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:17:53.340376 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Nov 02 07:17:53.340376 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:17:53.340376 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:17:53.340376 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:17:53.340376 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:17:53.340376 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:17:53.340376 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:17:53.340376 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:17:53.340376 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:17:53.341355 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Nov 02 07:17:53.341355 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Nov 02 07:17:53.341355 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Nov 02 07:17:53.341355 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Nov 02 07:17:53.341355 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Nov 02 07:17:53.341355 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Nov 02 07:17:53.341355 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 07:17:53.341355 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Nov 02 07:17:53.344068 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 371/743] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Nov 2 07:17:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 07:17:53.349581 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7121f631-0714-43bd-a338-1a2990c64e8e None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:53.360651 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10f133a-45ab-4802-843c-ae471ff87975 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:53.361636 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10f133a-45ab-4802-843c-ae471ff87975 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:53.361918 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10f133a-45ab-4802-843c-ae471ff87975 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:53.362192 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10f133a-45ab-4802-843c-ae471ff87975 None admin] PATH_INFO: `/v3/services` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:53.363184 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10f133a-45ab-4802-843c-ae471ff87975 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:17:53.364074 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10f133a-45ab-4802-843c-ae471ff87975 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:53.365443 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10f133a-45ab-4802-843c-ae471ff87975 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:53.371590 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 373/744] 10.222.0.18 () {56 vars in 1112 bytes} [Wed Nov 2 07:17:53 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:17:53.379981 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a427e1-74eb-44a3-bef6-efb7b88d1640 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:53.379981 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a427e1-74eb-44a3-bef6-efb7b88d1640 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:53.379981 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a427e1-74eb-44a3-bef6-efb7b88d1640 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:53.379981 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 374/745] 10.222.0.18 () {54 vars in 927 bytes} [Wed Nov 2 07:17:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:17:53.380825 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11a6633-60eb-441d-ad74-ecad3893a3c7 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:53.381031 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11a6633-60eb-441d-ad74-ecad3893a3c7 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:53.381303 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11a6633-60eb-441d-ad74-ecad3893a3c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:53.399974 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-f11a6633-60eb-441d-ad74-ecad3893a3c7 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:17:53.401146 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-f11a6633-60eb-441d-ad74-ecad3893a3c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:17:53.460982 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:53.461428 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7121f631-0714-43bd-a338-1a2990c64e8e None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:53.474757 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 375/746] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Nov 2 07:17:53 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Nov 02 07:17:53.474757 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11a6633-60eb-441d-ad74-ecad3893a3c7 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:53.505691 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b39db9c-1070-4434-8caa-742d815db29b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:53.506308 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b39db9c-1070-4434-8caa-742d815db29b None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:53.506620 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b39db9c-1070-4434-8caa-742d815db29b None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:53.506890 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b39db9c-1070-4434-8caa-742d815db29b None admin] PATH_INFO: `/v3/projects` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:53.508004 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b39db9c-1070-4434-8caa-742d815db29b None admin] RBAC: Adding query filter params (name=service) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:17:53.508322 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b39db9c-1070-4434-8caa-742d815db29b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:53.510104 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b39db9c-1070-4434-8caa-742d815db29b None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:53.546390 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 372/747] 10.222.0.18 () {56 vars in 1112 bytes} [Wed Nov 2 07:17:53 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:17:53.588645 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11a6633-60eb-441d-ad74-ecad3893a3c7 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:53.628030 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad9afc4-b1c0-4c70-8862-208897de6c5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:53.628714 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad9afc4-b1c0-4c70-8862-208897de6c5e None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:53.629594 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad9afc4-b1c0-4c70-8862-208897de6c5e None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:53.629868 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad9afc4-b1c0-4c70-8862-208897de6c5e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:53.630606 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad9afc4-b1c0-4c70-8862-208897de6c5e None admin] RBAC: Adding query filter params (interface=public, service_id=3e488cf0ba9b47f88a50558cbb2d59cc, region_id=RegionOne) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:17:53.630912 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad9afc4-b1c0-4c70-8862-208897de6c5e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:53.632299 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad9afc4-b1c0-4c70-8862-208897de6c5e None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:53.638684 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 376/748] 10.222.0.18 () {56 vars in 1251 bytes} [Wed Nov 2 07:17:53 2022] GET /identity/v3/endpoints?service_id=3e488cf0ba9b47f88a50558cbb2d59cc&interface=public®ion_id=RegionOne => generated 194 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:17:53.647628 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b39db9c-1070-4434-8caa-742d815db29b None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:53.679866 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:53.743968 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad9afc4-b1c0-4c70-8862-208897de6c5e None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:53.763767 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b39db9c-1070-4434-8caa-742d815db29b None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:53.806443 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e6a056-6e72-4ee9-ae4c-d045b236fdcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:53.807430 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e6a056-6e72-4ee9-ae4c-d045b236fdcd None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:53.807770 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e6a056-6e72-4ee9-ae4c-d045b236fdcd None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:53.808158 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e6a056-6e72-4ee9-ae4c-d045b236fdcd None admin] PATH_INFO: `/v3/services` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:53.808844 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e6a056-6e72-4ee9-ae4c-d045b236fdcd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:53.811709 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e6a056-6e72-4ee9-ae4c-d045b236fdcd None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:53.811830 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:53.819332 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 373/749] 10.222.0.18 () {56 vars in 1087 bytes} [Wed Nov 2 07:17:53 2022] GET /identity/v3/services => generated 1957 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:17:53.862394 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad9afc4-b1c0-4c70-8862-208897de6c5e None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:53.901136 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f865d3f8-d156-4dd3-ba5b-24b9e0cbf578 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:53.901852 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f865d3f8-d156-4dd3-ba5b-24b9e0cbf578 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:53.902135 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f865d3f8-d156-4dd3-ba5b-24b9e0cbf578 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:53.902429 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f865d3f8-d156-4dd3-ba5b-24b9e0cbf578 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:53.903086 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f865d3f8-d156-4dd3-ba5b-24b9e0cbf578 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:53.904521 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f865d3f8-d156-4dd3-ba5b-24b9e0cbf578 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:53.915158 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-f865d3f8-d156-4dd3-ba5b-24b9e0cbf578 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Nov 02 07:17:53.915158 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:17:53.915158 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Nov 02 07:17:53.915158 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Nov 02 07:17:53.915158 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Nov 02 07:17:53.915158 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Nov 02 07:17:53.915158 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Nov 02 07:17:53.915158 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:17:53.915158 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Nov 02 07:17:53.915158 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:17:53.915158 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 07:17:53.915158 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:17:53.915158 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:17:53.915158 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:17:53.915158 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:17:53.915158 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:17:53.915158 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:17:53.916220 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:17:53.916220 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:17:53.916220 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:17:53.916220 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:17:53.916220 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:17:53.916220 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:17:53.916220 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Nov 02 07:17:53.916220 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_service(service_id) Nov 02 07:17:53.916220 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Nov 02 07:17:53.916220 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Nov 02 07:17:53.916220 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:17:53.916220 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:17:53.916220 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:17:53.916220 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:17:53.916220 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:17:53.916220 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:17:53.917082 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:17:53.917082 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:17:53.917082 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:17:53.917082 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:17:53.917082 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:17:53.917082 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:17:53.917082 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:17:53.917082 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:17:53.917082 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:17:53.917082 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:17:53.917082 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Nov 02 07:17:53.917082 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:17:53.917082 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Nov 02 07:17:53.917082 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:17:53.917082 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 377/750] 10.222.0.18 () {56 vars in 1126 bytes} [Wed Nov 2 07:17:53 2022] GET /identity/v3/services/object-store => generated 93 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 07:17:53.924475 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e6a056-6e72-4ee9-ae4c-d045b236fdcd None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:54.021379 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:54.036145 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e6a056-6e72-4ee9-ae4c-d045b236fdcd None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:54.071907 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e9ac53-73d0-4ef1-93a7-3a81ab186f21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:54.072561 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e9ac53-73d0-4ef1-93a7-3a81ab186f21 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:54.072789 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e9ac53-73d0-4ef1-93a7-3a81ab186f21 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:54.073037 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e9ac53-73d0-4ef1-93a7-3a81ab186f21 None admin] PATH_INFO: `/v3/services` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:54.073655 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e9ac53-73d0-4ef1-93a7-3a81ab186f21 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:17:54.073952 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e9ac53-73d0-4ef1-93a7-3a81ab186f21 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:54.075351 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e9ac53-73d0-4ef1-93a7-3a81ab186f21 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:54.080737 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 374/751] 10.222.0.18 () {56 vars in 1122 bytes} [Wed Nov 2 07:17:53 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:17:54.086912 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f865d3f8-d156-4dd3-ba5b-24b9e0cbf578 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:54.192665 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f865d3f8-d156-4dd3-ba5b-24b9e0cbf578 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:54.215488 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:54.227639 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2e2dfe-cebd-409c-8be6-e7749216750d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:54.228508 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2e2dfe-cebd-409c-8be6-e7749216750d None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:54.228784 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2e2dfe-cebd-409c-8be6-e7749216750d None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:54.229048 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2e2dfe-cebd-409c-8be6-e7749216750d None admin] PATH_INFO: `/v3/services` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:54.229668 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2e2dfe-cebd-409c-8be6-e7749216750d None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:17:54.229961 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2e2dfe-cebd-409c-8be6-e7749216750d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:54.231330 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2e2dfe-cebd-409c-8be6-e7749216750d None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:54.236615 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 378/752] 10.222.0.18 () {56 vars in 1122 bytes} [Wed Nov 2 07:17:54 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:17:54.242705 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e9ac53-73d0-4ef1-93a7-3a81ab186f21 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:54.343938 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e9ac53-73d0-4ef1-93a7-3a81ab186f21 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:54.380245 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4e0469-9e9a-4a71-bd56-9e967fa589b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:54.380854 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4e0469-9e9a-4a71-bd56-9e967fa589b2 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:54.381127 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4e0469-9e9a-4a71-bd56-9e967fa589b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:54.381394 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4e0469-9e9a-4a71-bd56-9e967fa589b2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:54.382083 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4e0469-9e9a-4a71-bd56-9e967fa589b2 None admin] RBAC: Adding query filter params (interface=admin, service_id=508c32ed86cf420483d2aeffbb07c27a, region_id=RegionOne) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:17:54.382404 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4e0469-9e9a-4a71-bd56-9e967fa589b2 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:54.383893 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4e0469-9e9a-4a71-bd56-9e967fa589b2 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:54.389531 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 375/753] 10.222.0.18 () {56 vars in 1249 bytes} [Wed Nov 2 07:17:54 2022] GET /identity/v3/endpoints?service_id=508c32ed86cf420483d2aeffbb07c27a&interface=admin®ion_id=RegionOne => generated 193 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:17:54.395312 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2e2dfe-cebd-409c-8be6-e7749216750d None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:54.500828 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2e2dfe-cebd-409c-8be6-e7749216750d None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:54.536279 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e3682f-58bf-4b91-93d1-a8ef667a33af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:54.537250 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e3682f-58bf-4b91-93d1-a8ef667a33af None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:54.537701 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e3682f-58bf-4b91-93d1-a8ef667a33af None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:54.538149 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e3682f-58bf-4b91-93d1-a8ef667a33af None admin] PATH_INFO: `/v3/services` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:54.538945 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e3682f-58bf-4b91-93d1-a8ef667a33af None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:54.540448 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e3682f-58bf-4b91-93d1-a8ef667a33af None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:54.547504 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 379/754] 10.222.0.18 () {56 vars in 1087 bytes} [Wed Nov 2 07:17:54 2022] GET /identity/v3/services => generated 1957 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:17:54.899883 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ee60dc-ef13-4b7c-8f5a-b9326373f416 None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:54.900238 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ee60dc-ef13-4b7c-8f5a-b9326373f416 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:54.900341 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ee60dc-ef13-4b7c-8f5a-b9326373f416 None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:54.901915 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 376/755] 10.222.0.18 () {54 vars in 927 bytes} [Wed Nov 2 07:17:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:17:54.907430 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d085923-fee6-4274-80b6-6fe69c197bd8 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:54.907714 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d085923-fee6-4274-80b6-6fe69c197bd8 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:54.907985 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d085923-fee6-4274-80b6-6fe69c197bd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:54.927046 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-7d085923-fee6-4274-80b6-6fe69c197bd8 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:17:54.928218 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-7d085923-fee6-4274-80b6-6fe69c197bd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:17:54.974898 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:54.997855 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 380/756] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Nov 2 07:17:54 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 07:17:55.281347 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ee60dc-ef13-4b7c-8f5a-b9326373f416 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:55.298184 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:55.387940 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ee60dc-ef13-4b7c-8f5a-b9326373f416 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:55.422692 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce43f420-186a-4de2-a13b-e5bb7b13a4cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:55.423321 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce43f420-186a-4de2-a13b-e5bb7b13a4cb None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:55.423596 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce43f420-186a-4de2-a13b-e5bb7b13a4cb None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:55.423861 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce43f420-186a-4de2-a13b-e5bb7b13a4cb None admin] PATH_INFO: `/v3/services/image` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:55.424435 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce43f420-186a-4de2-a13b-e5bb7b13a4cb None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:55.426322 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce43f420-186a-4de2-a13b-e5bb7b13a4cb None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:55.435186 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-ce43f420-186a-4de2-a13b-e5bb7b13a4cb None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Nov 02 07:17:55.435186 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:17:55.435186 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Nov 02 07:17:55.435186 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Nov 02 07:17:55.435186 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Nov 02 07:17:55.435186 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Nov 02 07:17:55.435186 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Nov 02 07:17:55.435186 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:17:55.435186 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Nov 02 07:17:55.435186 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Nov 02 07:17:55.435186 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 07:17:55.435186 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Nov 02 07:17:55.435186 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:17:55.435186 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:17:55.435186 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:17:55.435186 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:17:55.435186 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:17:55.436163 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:17:55.436163 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:17:55.436163 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:17:55.436163 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:17:55.436163 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:17:55.436163 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:17:55.436163 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Nov 02 07:17:55.436163 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_service(service_id) Nov 02 07:17:55.436163 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Nov 02 07:17:55.436163 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Nov 02 07:17:55.436163 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:17:55.436163 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:17:55.436163 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:17:55.436163 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:17:55.436163 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:17:55.436163 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:17:55.437010 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:17:55.437010 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Nov 02 07:17:55.437010 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:17:55.437010 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:17:55.437010 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:17:55.437010 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:17:55.437010 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:17:55.437010 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:17:55.437010 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:17:55.437010 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:17:55.437010 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Nov 02 07:17:55.437010 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:17:55.437010 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Nov 02 07:17:55.437010 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Nov 02 07:17:55.437010 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 377/757] 10.222.0.18 () {56 vars in 1105 bytes} [Wed Nov 2 07:17:55 2022] GET /identity/v3/services/image => generated 86 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:17:55.442955 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d085923-fee6-4274-80b6-6fe69c197bd8 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:55.549801 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d085923-fee6-4274-80b6-6fe69c197bd8 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:55.584986 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709df397-6406-4df3-9231-3eebd7f14cfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:55.585592 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709df397-6406-4df3-9231-3eebd7f14cfc None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:55.585870 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709df397-6406-4df3-9231-3eebd7f14cfc None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:55.586140 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709df397-6406-4df3-9231-3eebd7f14cfc None admin] PATH_INFO: `/v3/services` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:55.586794 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709df397-6406-4df3-9231-3eebd7f14cfc None admin] RBAC: Adding query filter params (name=image) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:17:55.587096 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709df397-6406-4df3-9231-3eebd7f14cfc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:55.588450 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709df397-6406-4df3-9231-3eebd7f14cfc None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:55.593709 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 381/758] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Nov 2 07:17:55 2022] GET /identity/v3/services?name=image => generated 122 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:17:55.599999 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce43f420-186a-4de2-a13b-e5bb7b13a4cb None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:55.683442 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:55.710513 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce43f420-186a-4de2-a13b-e5bb7b13a4cb None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:55.747255 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b58d6e2-7646-4d7c-9e6f-26ca76be5b58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:55.748250 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b58d6e2-7646-4d7c-9e6f-26ca76be5b58 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:55.748737 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b58d6e2-7646-4d7c-9e6f-26ca76be5b58 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:55.749214 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b58d6e2-7646-4d7c-9e6f-26ca76be5b58 None admin] PATH_INFO: `/v3/services` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:55.750038 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b58d6e2-7646-4d7c-9e6f-26ca76be5b58 None admin] RBAC: Adding query filter params (type=image) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:17:55.750574 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b58d6e2-7646-4d7c-9e6f-26ca76be5b58 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:55.752177 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b58d6e2-7646-4d7c-9e6f-26ca76be5b58 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:55.757962 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 378/759] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Nov 2 07:17:55 2022] GET /identity/v3/services?type=image => generated 350 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:17:55.763555 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709df397-6406-4df3-9231-3eebd7f14cfc None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:55.812492 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:55.879832 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709df397-6406-4df3-9231-3eebd7f14cfc None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:55.917120 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444e5477-d581-4d9c-ac57-199b4e4f9c43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:55.918119 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444e5477-d581-4d9c-ac57-199b4e4f9c43 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:55.918632 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444e5477-d581-4d9c-ac57-199b4e4f9c43 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:55.919108 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444e5477-d581-4d9c-ac57-199b4e4f9c43 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:55.920007 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444e5477-d581-4d9c-ac57-199b4e4f9c43 None admin] RBAC: Adding query filter params (interface=public, service_id=c3c8fc5acdd640479d229662a701a611, region_id=RegionOne) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:17:55.920532 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444e5477-d581-4d9c-ac57-199b4e4f9c43 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:55.922296 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444e5477-d581-4d9c-ac57-199b4e4f9c43 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:55.928465 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 382/760] 10.222.0.18 () {56 vars in 1251 bytes} [Wed Nov 2 07:17:55 2022] GET /identity/v3/endpoints?service_id=c3c8fc5acdd640479d229662a701a611&interface=public®ion_id=RegionOne => generated 194 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:17:55.935742 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b58d6e2-7646-4d7c-9e6f-26ca76be5b58 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:55.936703 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fcf47a-a4e3-4b4e-a562-8d58331cbe2f None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:55.936990 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fcf47a-a4e3-4b4e-a562-8d58331cbe2f None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:55.937268 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fcf47a-a4e3-4b4e-a562-8d58331cbe2f None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:55.938867 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 383/761] 10.222.0.18 () {54 vars in 927 bytes} [Wed Nov 2 07:17:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Nov 02 07:17:55.944222 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28589ca3-94dd-41f9-bc03-8269454b4fd4 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:55.946407 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28589ca3-94dd-41f9-bc03-8269454b4fd4 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:55.946407 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28589ca3-94dd-41f9-bc03-8269454b4fd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:55.963784 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-28589ca3-94dd-41f9-bc03-8269454b4fd4 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:17:55.965045 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-28589ca3-94dd-41f9-bc03-8269454b4fd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:17:56.046088 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 384/762] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Nov 2 07:17:55 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Nov 02 07:17:56.075112 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b58d6e2-7646-4d7c-9e6f-26ca76be5b58 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:56.111412 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb067a8-e2d6-4f20-9ceb-6cdadf0d5933 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:56.112417 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb067a8-e2d6-4f20-9ceb-6cdadf0d5933 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:56.112877 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb067a8-e2d6-4f20-9ceb-6cdadf0d5933 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:56.113877 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb067a8-e2d6-4f20-9ceb-6cdadf0d5933 None admin] PATH_INFO: `/v3/services` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:56.114683 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb067a8-e2d6-4f20-9ceb-6cdadf0d5933 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:56.116236 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb067a8-e2d6-4f20-9ceb-6cdadf0d5933 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:56.120232 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b8dc03-e6f0-4799-bb60-fd8e0698b12d None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:56.120512 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b8dc03-e6f0-4799-bb60-fd8e0698b12d None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:56.120785 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b8dc03-e6f0-4799-bb60-fd8e0698b12d None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:56.122347 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 385/763] 10.222.0.18 () {54 vars in 927 bytes} [Wed Nov 2 07:17:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Nov 02 07:17:56.124399 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 379/764] 10.222.0.18 () {56 vars in 1087 bytes} [Wed Nov 2 07:17:55 2022] GET /identity/v3/services => generated 1957 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:17:56.131435 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab2287a-15e4-481a-ab45-9ec5c820732c None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:56.131435 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab2287a-15e4-481a-ab45-9ec5c820732c None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:56.131435 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab2287a-15e4-481a-ab45-9ec5c820732c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:56.147442 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-0ab2287a-15e4-481a-ab45-9ec5c820732c None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:17:56.148661 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-0ab2287a-15e4-481a-ab45-9ec5c820732c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:17:56.184295 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:56.227198 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 386/765] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Nov 2 07:17:56 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 07:17:56.344103 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb067a8-e2d6-4f20-9ceb-6cdadf0d5933 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:56.355232 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:56.454986 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb067a8-e2d6-4f20-9ceb-6cdadf0d5933 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:56.490989 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b53a904-22e5-4453-b59c-d4fca70fe488 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:56.491599 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b53a904-22e5-4453-b59c-d4fca70fe488 None admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:56.491879 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b53a904-22e5-4453-b59c-d4fca70fe488 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:56.492149 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b53a904-22e5-4453-b59c-d4fca70fe488 None admin] PATH_INFO: `/v3/users` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:56.492793 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b53a904-22e5-4453-b59c-d4fca70fe488 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:56.494201 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b53a904-22e5-4453-b59c-d4fca70fe488 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:56.511701 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.sql.core [None req-8b53a904-22e5-4453-b59c-d4fca70fe488 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Nov 02 07:17:56.511701 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Nov 02 07:17:56.511701 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [parameters: {'user_id': 'b3268caa51df484186b927fcf2a391ff', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Nov 02 07:17:56.511701 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81501) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Nov 02 07:17:56.514990 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab2287a-15e4-481a-ab45-9ec5c820732c None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:56.519265 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-8b53a904-22e5-4453-b59c-d4fca70fe488 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Nov 02 07:17:56.519265 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:17:56.519265 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Nov 02 07:17:56.519265 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application self.dialect.do_execute( Nov 02 07:17:56.519265 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Nov 02 07:17:56.519265 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Nov 02 07:17:56.519265 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Nov 02 07:17:56.519265 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application result = self._query(query) Nov 02 07:17:56.519265 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Nov 02 07:17:56.519265 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application conn.query(q) Nov 02 07:17:56.519265 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Nov 02 07:17:56.519265 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Nov 02 07:17:56.519265 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Nov 02 07:17:56.519265 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application result.read() Nov 02 07:17:56.519265 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Nov 02 07:17:56.520581 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Nov 02 07:17:56.520581 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Nov 02 07:17:56.520581 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application packet.raise_for_error() Nov 02 07:17:56.520581 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Nov 02 07:17:56.520581 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Nov 02 07:17:56.520581 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Nov 02 07:17:56.520581 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Nov 02 07:17:56.520581 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Nov 02 07:17:56.520581 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Nov 02 07:17:56.520581 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Nov 02 07:17:56.520581 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Nov 02 07:17:56.520581 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:17:56.520581 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Nov 02 07:17:56.520581 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return method(*args, **kwargs) Nov 02 07:17:56.520581 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Nov 02 07:17:56.520581 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Nov 02 07:17:56.520581 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Nov 02 07:17:56.520581 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application next(self.gen) Nov 02 07:17:56.521528 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1068, in _transaction_scope Nov 02 07:17:56.521528 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application yield resource Nov 02 07:17:56.521528 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Nov 02 07:17:56.521528 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application next(self.gen) Nov 02 07:17:56.521528 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 670, in _session Nov 02 07:17:56.521528 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application self.session.rollback() Nov 02 07:17:56.521528 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Nov 02 07:17:56.521528 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application self.force_reraise() Nov 02 07:17:56.521528 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Nov 02 07:17:56.521528 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise self.value Nov 02 07:17:56.521528 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 667, in _session Nov 02 07:17:56.521528 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Nov 02 07:17:56.521528 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 695, in _end_session_transaction Nov 02 07:17:56.521528 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application session.commit() Nov 02 07:17:56.521528 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Nov 02 07:17:56.521528 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Nov 02 07:17:56.521528 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Nov 02 07:17:56.522460 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application self._prepare_impl() Nov 02 07:17:56.522460 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Nov 02 07:17:56.522460 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application self.session.flush() Nov 02 07:17:56.522460 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Nov 02 07:17:56.522460 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application self._flush(objects) Nov 02 07:17:56.522460 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Nov 02 07:17:56.522460 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Nov 02 07:17:56.522460 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Nov 02 07:17:56.522460 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application compat.raise_( Nov 02 07:17:56.522460 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Nov 02 07:17:56.522460 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception Nov 02 07:17:56.522460 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Nov 02 07:17:56.522460 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application flush_context.execute() Nov 02 07:17:56.522460 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Nov 02 07:17:56.522460 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rec.execute(self) Nov 02 07:17:56.522460 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Nov 02 07:17:56.522460 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Nov 02 07:17:56.523359 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Nov 02 07:17:56.523359 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application _emit_insert_statements( Nov 02 07:17:56.523359 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Nov 02 07:17:56.523359 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application result = connection._execute_20( Nov 02 07:17:56.523359 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Nov 02 07:17:56.523359 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Nov 02 07:17:56.523359 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Nov 02 07:17:56.523359 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Nov 02 07:17:56.523359 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Nov 02 07:17:56.523359 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application ret = self._execute_context( Nov 02 07:17:56.523359 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Nov 02 07:17:56.523359 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Nov 02 07:17:56.523359 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Nov 02 07:17:56.523359 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Nov 02 07:17:56.523359 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Nov 02 07:17:56.523359 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception Nov 02 07:17:56.524220 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Nov 02 07:17:56.524220 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application self.dialect.do_execute( Nov 02 07:17:56.524220 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Nov 02 07:17:56.524220 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Nov 02 07:17:56.524220 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Nov 02 07:17:56.524220 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application result = self._query(query) Nov 02 07:17:56.524220 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Nov 02 07:17:56.524220 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application conn.query(q) Nov 02 07:17:56.524220 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Nov 02 07:17:56.524220 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Nov 02 07:17:56.524220 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Nov 02 07:17:56.524220 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application result.read() Nov 02 07:17:56.524220 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Nov 02 07:17:56.524220 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Nov 02 07:17:56.524220 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Nov 02 07:17:56.524220 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application packet.raise_for_error() Nov 02 07:17:56.525025 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Nov 02 07:17:56.525025 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Nov 02 07:17:56.525025 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Nov 02 07:17:56.525025 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Nov 02 07:17:56.525025 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Nov 02 07:17:56.525025 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Nov 02 07:17:56.525025 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [parameters: {'user_id': 'b3268caa51df484186b927fcf2a391ff', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Nov 02 07:17:56.525025 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Nov 02 07:17:56.525025 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Nov 02 07:17:56.525025 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 07:17:56.525025 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Nov 02 07:17:56.525025 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:17:56.525025 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:17:56.525025 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:17:56.525025 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:17:56.525815 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:17:56.525815 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:17:56.525815 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:17:56.525815 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:17:56.525815 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:17:56.525815 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:17:56.525815 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:17:56.525815 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Nov 02 07:17:56.525815 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Nov 02 07:17:56.525815 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:17:56.525815 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:17:56.525815 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Nov 02 07:17:56.525815 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:17:56.525815 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Nov 02 07:17:56.525815 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:17:56.525815 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Nov 02 07:17:56.525815 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Nov 02 07:17:56.533139 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Nov 02 07:17:56.533139 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Nov 02 07:17:56.533139 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Nov 02 07:17:56.533139 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Nov 02 07:17:56.533139 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Nov 02 07:17:56.533139 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Nov 02 07:17:56.533139 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 380/766] 10.222.0.18 () {60 vars in 1131 bytes} [Wed Nov 2 07:17:56 2022] POST /identity/v3/users => generated 160 bytes in 177 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:17:56.533139 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b53a904-22e5-4453-b59c-d4fca70fe488 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:56.604607 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:56.635394 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab2287a-15e4-481a-ab45-9ec5c820732c None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:56.643023 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b53a904-22e5-4453-b59c-d4fca70fe488 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:56.674467 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd871e3a-8651-470e-b070-40049e7249cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:56.675363 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd871e3a-8651-470e-b070-40049e7249cc None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:56.675914 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd871e3a-8651-470e-b070-40049e7249cc None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:56.676464 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd871e3a-8651-470e-b070-40049e7249cc None admin] PATH_INFO: `/v3/services/compute` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:56.677269 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd871e3a-8651-470e-b070-40049e7249cc None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:56.678883 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd871e3a-8651-470e-b070-40049e7249cc None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:56.679144 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb376ddd-5f70-4abf-8119-71cfa3aeda4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:56.679961 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb376ddd-5f70-4abf-8119-71cfa3aeda4e None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:56.680247 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb376ddd-5f70-4abf-8119-71cfa3aeda4e None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:56.680517 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb376ddd-5f70-4abf-8119-71cfa3aeda4e None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:56.688335 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb376ddd-5f70-4abf-8119-71cfa3aeda4e None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:56.689566 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-bd871e3a-8651-470e-b070-40049e7249cc None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Nov 02 07:17:56.689566 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:17:56.689566 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Nov 02 07:17:56.689566 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Nov 02 07:17:56.689566 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Nov 02 07:17:56.689566 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Nov 02 07:17:56.689566 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Nov 02 07:17:56.689566 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:17:56.689566 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Nov 02 07:17:56.689566 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:17:56.689566 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 07:17:56.689566 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:17:56.689566 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:17:56.689566 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:17:56.689566 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:17:56.689566 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:17:56.689566 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:17:56.690560 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:17:56.690560 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:17:56.690560 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:17:56.690560 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:17:56.690560 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:17:56.690560 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:17:56.690560 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Nov 02 07:17:56.690560 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_service(service_id) Nov 02 07:17:56.690560 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Nov 02 07:17:56.690560 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Nov 02 07:17:56.690560 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:17:56.690560 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:17:56.690560 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:17:56.690560 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:17:56.690560 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:17:56.690560 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:17:56.691933 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:17:56.691933 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:17:56.691933 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:17:56.691933 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:17:56.691933 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:17:56.691933 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:17:56.691933 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:17:56.691933 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:17:56.691933 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:17:56.691933 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:17:56.691933 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Nov 02 07:17:56.691933 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:17:56.691933 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Nov 02 07:17:56.691933 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:17:56.692688 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb376ddd-5f70-4abf-8119-71cfa3aeda4e None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:56.699728 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 387/767] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Nov 2 07:17:56 2022] GET /identity/v3/services/compute => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 07:17:56.700839 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd871e3a-8651-470e-b070-40049e7249cc None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:56.702755 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-bb376ddd-5f70-4abf-8119-71cfa3aeda4e None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Nov 02 07:17:56.702755 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:17:56.702755 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:17:56.702755 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:17:56.702755 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:17:56.702755 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:17:56.702755 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:17:56.702755 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:17:56.702755 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:17:56.702755 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:17:56.702755 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:17:56.702755 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:17:56.702755 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Nov 02 07:17:56.702755 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_user(user_id) Nov 02 07:17:56.702755 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Nov 02 07:17:56.702755 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Nov 02 07:17:56.703693 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:17:56.703693 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:17:56.703693 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Nov 02 07:17:56.703693 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:17:56.703693 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Nov 02 07:17:56.703693 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:17:56.703693 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:17:56.703693 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:17:56.703693 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:17:56.703693 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:17:56.703693 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:17:56.703693 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Nov 02 07:17:56.703693 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:17:56.703693 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:17:56.703693 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:17:56.703693 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:17:56.703693 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:17:56.703693 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:17:56.704725 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:17:56.704725 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:17:56.704725 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Nov 02 07:17:56.704725 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Nov 02 07:17:56.704725 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Nov 02 07:17:56.704725 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Nov 02 07:17:56.704725 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Nov 02 07:17:56.704725 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Nov 02 07:17:56.704725 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Nov 02 07:17:56.704725 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Nov 02 07:17:56.708931 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 381/768] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Nov 2 07:17:56 2022] GET /identity/v3/users/cinder => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:17:56.709894 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb376ddd-5f70-4abf-8119-71cfa3aeda4e None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:56.753081 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:56.817601 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb376ddd-5f70-4abf-8119-71cfa3aeda4e None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:56.818487 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd871e3a-8651-470e-b070-40049e7249cc None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:56.845269 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:56.854143 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1dba0c-5113-4d67-ba70-e08261add373 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:56.854837 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1dba0c-5113-4d67-ba70-e08261add373 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:56.855068 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1dba0c-5113-4d67-ba70-e08261add373 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:56.855334 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1dba0c-5113-4d67-ba70-e08261add373 None admin] PATH_INFO: `/v3/users` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:56.856018 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1dba0c-5113-4d67-ba70-e08261add373 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:17:56.856314 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1dba0c-5113-4d67-ba70-e08261add373 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:56.857180 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d921eae7-a264-4821-8778-95eae338348a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:56.857723 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1dba0c-5113-4d67-ba70-e08261add373 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:56.857819 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d921eae7-a264-4821-8778-95eae338348a None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:56.858056 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d921eae7-a264-4821-8778-95eae338348a None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:56.858328 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d921eae7-a264-4821-8778-95eae338348a None admin] PATH_INFO: `/v3/services` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:56.858989 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d921eae7-a264-4821-8778-95eae338348a None admin] RBAC: Adding query filter params (name=compute) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:17:56.859291 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d921eae7-a264-4821-8778-95eae338348a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:56.860628 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d921eae7-a264-4821-8778-95eae338348a None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:56.869098 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 388/769] 10.222.0.18 () {56 vars in 1112 bytes} [Wed Nov 2 07:17:56 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:17:56.869098 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8656ed6a-cb33-4a83-90c3-1b6fe217d03b None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:56.869098 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8656ed6a-cb33-4a83-90c3-1b6fe217d03b None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:56.869098 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8656ed6a-cb33-4a83-90c3-1b6fe217d03b None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:56.869098 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 389/770] 10.222.0.18 () {54 vars in 927 bytes} [Wed Nov 2 07:17:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:17:56.874487 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8656ed6a-cb33-4a83-90c3-1b6fe217d03b None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:56.906395 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 382/771] 10.222.0.18 () {56 vars in 1101 bytes} [Wed Nov 2 07:17:56 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:17:56.906395 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407b49d8-2035-4469-8b6d-639743f675a0 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:56.906395 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407b49d8-2035-4469-8b6d-639743f675a0 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:56.906395 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407b49d8-2035-4469-8b6d-639743f675a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:56.921041 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-407b49d8-2035-4469-8b6d-639743f675a0 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:17:56.922868 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-407b49d8-2035-4469-8b6d-639743f675a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:17:57.009475 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:57.014639 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 383/772] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Nov 2 07:17:56 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 07:17:57.022765 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8656ed6a-cb33-4a83-90c3-1b6fe217d03b None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:57.063461 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64157a5a-81c2-4f00-8ab6-8168caf9edca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:57.064455 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64157a5a-81c2-4f00-8ab6-8168caf9edca None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:57.064922 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64157a5a-81c2-4f00-8ab6-8168caf9edca None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:57.065393 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64157a5a-81c2-4f00-8ab6-8168caf9edca None admin] PATH_INFO: `/v3/services` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:57.066221 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64157a5a-81c2-4f00-8ab6-8168caf9edca None admin] RBAC: Adding query filter params (type=compute) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:17:57.067615 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64157a5a-81c2-4f00-8ab6-8168caf9edca None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:57.069211 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64157a5a-81c2-4f00-8ab6-8168caf9edca None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:57.075493 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 390/773] 10.222.0.18 () {56 vars in 1112 bytes} [Wed Nov 2 07:17:56 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:17:57.081424 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407b49d8-2035-4469-8b6d-639743f675a0 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:57.193508 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407b49d8-2035-4469-8b6d-639743f675a0 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:57.211417 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:57.229928 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d1c2ec-d9fa-4e46-ac6c-04ea3727e2a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:57.230797 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d1c2ec-d9fa-4e46-ac6c-04ea3727e2a8 None admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:57.231220 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d1c2ec-d9fa-4e46-ac6c-04ea3727e2a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:57.231635 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d1c2ec-d9fa-4e46-ac6c-04ea3727e2a8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:57.232444 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d1c2ec-d9fa-4e46-ac6c-04ea3727e2a8 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:57.233972 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d1c2ec-d9fa-4e46-ac6c-04ea3727e2a8 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:57.256291 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 384/774] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Nov 2 07:17:57 2022] POST /identity/v3/endpoints => generated 334 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:17:57.300042 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64157a5a-81c2-4f00-8ab6-8168caf9edca None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:57.413891 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64157a5a-81c2-4f00-8ab6-8168caf9edca None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:57.457728 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b61f879-8a64-4413-8611-b2de7eb84a06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:57.458286 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b61f879-8a64-4413-8611-b2de7eb84a06 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:57.458634 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b61f879-8a64-4413-8611-b2de7eb84a06 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:57.458904 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b61f879-8a64-4413-8611-b2de7eb84a06 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:57.459559 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b61f879-8a64-4413-8611-b2de7eb84a06 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:57.461037 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b61f879-8a64-4413-8611-b2de7eb84a06 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:57.469750 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-3b61f879-8a64-4413-8611-b2de7eb84a06 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Nov 02 07:17:57.469750 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:17:57.469750 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Nov 02 07:17:57.469750 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Nov 02 07:17:57.469750 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Nov 02 07:17:57.469750 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Nov 02 07:17:57.469750 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Nov 02 07:17:57.469750 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:17:57.469750 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Nov 02 07:17:57.469750 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:17:57.469750 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 07:17:57.469750 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:17:57.469750 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:17:57.469750 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:17:57.469750 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:17:57.469750 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:17:57.469750 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:17:57.470737 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:17:57.470737 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:17:57.470737 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:17:57.470737 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:17:57.470737 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:17:57.470737 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:17:57.470737 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Nov 02 07:17:57.470737 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_service(service_id) Nov 02 07:17:57.470737 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Nov 02 07:17:57.470737 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Nov 02 07:17:57.470737 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:17:57.470737 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:17:57.470737 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:17:57.470737 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:17:57.470737 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:17:57.470737 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:17:57.471930 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:17:57.471930 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:17:57.471930 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:17:57.471930 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:17:57.471930 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:17:57.471930 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:17:57.471930 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:17:57.471930 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:17:57.471930 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:17:57.471930 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:17:57.471930 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Nov 02 07:17:57.471930 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:17:57.471930 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Nov 02 07:17:57.471930 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:17:57.471930 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 391/775] 10.222.0.18 () {56 vars in 1126 bytes} [Wed Nov 2 07:17:57 2022] GET /identity/v3/services/object-store => generated 93 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:17:57.477954 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d1c2ec-d9fa-4e46-ac6c-04ea3727e2a8 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:57.580146 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d1c2ec-d9fa-4e46-ac6c-04ea3727e2a8 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:57.613487 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d9acf7-5643-4fb4-8710-9105cb9e89fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:57.614094 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d9acf7-5643-4fb4-8710-9105cb9e89fc None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:57.614400 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d9acf7-5643-4fb4-8710-9105cb9e89fc None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:57.614674 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d9acf7-5643-4fb4-8710-9105cb9e89fc None admin] PATH_INFO: `/v3/services` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:57.615297 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d9acf7-5643-4fb4-8710-9105cb9e89fc None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:17:57.615597 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d9acf7-5643-4fb4-8710-9105cb9e89fc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:57.617015 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d9acf7-5643-4fb4-8710-9105cb9e89fc None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:57.626531 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 385/776] 10.222.0.18 () {56 vars in 1122 bytes} [Wed Nov 2 07:17:57 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:17:57.627997 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b61f879-8a64-4413-8611-b2de7eb84a06 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:57.659997 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:57.739031 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b61f879-8a64-4413-8611-b2de7eb84a06 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:57.773244 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acacb6ac-3a11-4dab-aae7-9224fbd56de0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:57.774139 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acacb6ac-3a11-4dab-aae7-9224fbd56de0 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:57.774651 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acacb6ac-3a11-4dab-aae7-9224fbd56de0 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:57.775098 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acacb6ac-3a11-4dab-aae7-9224fbd56de0 None admin] PATH_INFO: `/v3/services` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:57.775903 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acacb6ac-3a11-4dab-aae7-9224fbd56de0 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:17:57.776379 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acacb6ac-3a11-4dab-aae7-9224fbd56de0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:57.777895 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acacb6ac-3a11-4dab-aae7-9224fbd56de0 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:57.783161 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 392/777] 10.222.0.18 () {56 vars in 1122 bytes} [Wed Nov 2 07:17:57 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:17:57.788788 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d9acf7-5643-4fb4-8710-9105cb9e89fc None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:57.828467 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:57.897418 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d9acf7-5643-4fb4-8710-9105cb9e89fc None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:57.930948 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4350a4-2ead-4180-b8a3-c2d5bca87bff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:57.931552 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4350a4-2ead-4180-b8a3-c2d5bca87bff None admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:57.931825 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4350a4-2ead-4180-b8a3-c2d5bca87bff None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:57.932089 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4350a4-2ead-4180-b8a3-c2d5bca87bff None admin] PATH_INFO: `/v3/endpoints` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:57.932752 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4350a4-2ead-4180-b8a3-c2d5bca87bff None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:57.934095 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4350a4-2ead-4180-b8a3-c2d5bca87bff None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:57.951409 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 386/778] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Nov 2 07:17:57 2022] POST /identity/v3/endpoints => generated 325 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:17:58.363834 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d802f57f-e20a-4634-9f2c-da02ccfdf855 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:58.364204 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d802f57f-e20a-4634-9f2c-da02ccfdf855 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:58.364386 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d802f57f-e20a-4634-9f2c-da02ccfdf855 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:58.365859 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 393/779] 10.222.0.18 () {54 vars in 927 bytes} [Wed Nov 2 07:17:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:17:58.370881 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173b3d1a-67e6-4397-896a-6a89e7499c0b None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:58.371175 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173b3d1a-67e6-4397-896a-6a89e7499c0b None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:58.371452 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173b3d1a-67e6-4397-896a-6a89e7499c0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:58.389943 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-173b3d1a-67e6-4397-896a-6a89e7499c0b None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:17:58.391160 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-173b3d1a-67e6-4397-896a-6a89e7499c0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:17:58.466234 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 387/780] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Nov 2 07:17:58 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 07:17:58.735095 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d802f57f-e20a-4634-9f2c-da02ccfdf855 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:58.820932 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:58.839104 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d802f57f-e20a-4634-9f2c-da02ccfdf855 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:58.875493 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d072e0-66eb-46a0-8e90-287a88e1849a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:58.876116 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d072e0-66eb-46a0-8e90-287a88e1849a None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:58.876390 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d072e0-66eb-46a0-8e90-287a88e1849a None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:58.876658 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d072e0-66eb-46a0-8e90-287a88e1849a None admin] PATH_INFO: `/v3/services/image` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:58.877226 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d072e0-66eb-46a0-8e90-287a88e1849a None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:58.878587 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d072e0-66eb-46a0-8e90-287a88e1849a None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:58.887289 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-85d072e0-66eb-46a0-8e90-287a88e1849a None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Nov 02 07:17:58.887289 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:17:58.887289 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Nov 02 07:17:58.887289 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Nov 02 07:17:58.887289 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Nov 02 07:17:58.887289 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Nov 02 07:17:58.887289 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Nov 02 07:17:58.887289 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:17:58.887289 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Nov 02 07:17:58.887289 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:17:58.887289 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 07:17:58.887289 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:17:58.887289 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:17:58.887289 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:17:58.887289 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:17:58.887289 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:17:58.887289 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:17:58.888281 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:17:58.888281 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:17:58.888281 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:17:58.888281 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:17:58.888281 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:17:58.888281 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:17:58.888281 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Nov 02 07:17:58.888281 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_service(service_id) Nov 02 07:17:58.888281 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Nov 02 07:17:58.888281 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Nov 02 07:17:58.888281 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:17:58.888281 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:17:58.888281 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:17:58.888281 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:17:58.888281 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:17:58.888281 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:17:58.889128 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:17:58.889128 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:17:58.889128 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:17:58.889128 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:17:58.889128 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:17:58.889128 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:17:58.889128 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:17:58.889128 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:17:58.889128 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:17:58.889128 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:17:58.889128 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Nov 02 07:17:58.889128 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:17:58.889128 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Nov 02 07:17:58.889128 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:17:58.893165 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 394/781] 10.222.0.18 () {56 vars in 1105 bytes} [Wed Nov 2 07:17:58 2022] GET /identity/v3/services/image => generated 86 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:17:58.894529 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173b3d1a-67e6-4397-896a-6a89e7499c0b None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:58.919359 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:58.997929 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173b3d1a-67e6-4397-896a-6a89e7499c0b None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:59.032013 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f32196-bc18-4bfc-aecd-5648d4ae95dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:59.032625 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f32196-bc18-4bfc-aecd-5648d4ae95dd None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:59.032906 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f32196-bc18-4bfc-aecd-5648d4ae95dd None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:59.033177 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f32196-bc18-4bfc-aecd-5648d4ae95dd None admin] PATH_INFO: `/v3/services` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:59.033800 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f32196-bc18-4bfc-aecd-5648d4ae95dd None admin] RBAC: Adding query filter params (name=image) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:17:59.034102 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f32196-bc18-4bfc-aecd-5648d4ae95dd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:59.036274 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f32196-bc18-4bfc-aecd-5648d4ae95dd None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:59.045709 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 388/782] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Nov 2 07:17:58 2022] GET /identity/v3/services?name=image => generated 122 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:17:59.047260 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d072e0-66eb-46a0-8e90-287a88e1849a None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:59.153132 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d072e0-66eb-46a0-8e90-287a88e1849a None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:59.188642 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313282e6-8b1e-48b9-b6ce-47b32fe65a9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:59.189156 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313282e6-8b1e-48b9-b6ce-47b32fe65a9a None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:59.189432 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313282e6-8b1e-48b9-b6ce-47b32fe65a9a None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:59.189697 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313282e6-8b1e-48b9-b6ce-47b32fe65a9a None admin] PATH_INFO: `/v3/services` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:59.190313 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313282e6-8b1e-48b9-b6ce-47b32fe65a9a None admin] RBAC: Adding query filter params (type=image) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:17:59.190633 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313282e6-8b1e-48b9-b6ce-47b32fe65a9a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:59.192093 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313282e6-8b1e-48b9-b6ce-47b32fe65a9a None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:59.197110 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 395/783] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Nov 2 07:17:59 2022] GET /identity/v3/services?type=image => generated 350 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:17:59.202681 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f32196-bc18-4bfc-aecd-5648d4ae95dd None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:59.293799 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:59.310311 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f32196-bc18-4bfc-aecd-5648d4ae95dd None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:59.345571 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33956698-104b-4184-a824-3bd570d9350e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:59.346183 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33956698-104b-4184-a824-3bd570d9350e None admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:59.346503 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33956698-104b-4184-a824-3bd570d9350e None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:59.346777 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33956698-104b-4184-a824-3bd570d9350e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:59.347457 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33956698-104b-4184-a824-3bd570d9350e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:59.349562 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33956698-104b-4184-a824-3bd570d9350e None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:59.374409 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 389/784] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Nov 2 07:17:59 2022] POST /identity/v3/endpoints => generated 327 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:17:59.505856 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dee09b-57aa-4efc-be90-d9619d4b98e1 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:59.506207 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dee09b-57aa-4efc-be90-d9619d4b98e1 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:59.506442 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dee09b-57aa-4efc-be90-d9619d4b98e1 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:59.508014 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 396/785] 10.222.0.18 () {54 vars in 927 bytes} [Wed Nov 2 07:17:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:17:59.513504 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f962f85d-d19b-4e97-9c73-3ef0e2bd0f57 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:59.513795 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f962f85d-d19b-4e97-9c73-3ef0e2bd0f57 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:59.514061 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f962f85d-d19b-4e97-9c73-3ef0e2bd0f57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:59.533897 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-f962f85d-d19b-4e97-9c73-3ef0e2bd0f57 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:17:59.535745 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-f962f85d-d19b-4e97-9c73-3ef0e2bd0f57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:17:59.613359 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 390/786] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Nov 2 07:17:59 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 07:17:59.879973 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0dee09b-57aa-4efc-be90-d9619d4b98e1 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:59.895543 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:59.983748 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0dee09b-57aa-4efc-be90-d9619d4b98e1 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:00.016474 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da94196-f13e-4a18-9372-88873e01e55b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:00.017097 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da94196-f13e-4a18-9372-88873e01e55b None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:00.017368 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da94196-f13e-4a18-9372-88873e01e55b None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:00.017633 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da94196-f13e-4a18-9372-88873e01e55b None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:00.030471 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da94196-f13e-4a18-9372-88873e01e55b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:18:00.036391 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-5da94196-f13e-4a18-9372-88873e01e55b None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Nov 02 07:18:00.036391 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:18:00.036391 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:18:00.036391 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:18:00.036391 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:18:00.036391 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:18:00.036391 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:18:00.036391 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:18:00.036391 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:18:00.036391 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:18:00.036391 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:18:00.036391 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:18:00.036391 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:18:00.036391 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:18:00.036391 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:18:00.036391 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:18:00.037541 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:18:00.037541 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:18:00.037541 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:18:00.037541 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:18:00.037541 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:18:00.037541 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:18:00.037541 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:18:00.037541 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:18:00.037541 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:18:00.037541 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:18:00.037541 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:18:00.037541 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:18:00.037541 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:18:00.037541 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:18:00.037541 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:18:00.037541 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:18:00.038411 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:18:00.038411 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:18:00.038411 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:18:00.038411 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:18:00.038411 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:18:00.038411 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:18:00.038411 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:18:00.038411 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:18:00.038411 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:18:00.038411 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:18:00.038411 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Nov 02 07:18:00.038411 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:18:00.042467 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 397/787] 10.222.0.18 () {56 vars in 1120 bytes} [Wed Nov 2 07:17:59 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:18:00.043903 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f962f85d-d19b-4e97-9c73-3ef0e2bd0f57 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:00.115909 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:18:00.150044 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b19cab0-bed2-4798-b98f-8b7bdd110c5c None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:00.150223 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b19cab0-bed2-4798-b98f-8b7bdd110c5c None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:00.150542 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b19cab0-bed2-4798-b98f-8b7bdd110c5c None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:00.152031 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 398/788] 10.222.0.18 () {54 vars in 927 bytes} [Wed Nov 2 07:18:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Nov 02 07:18:00.155300 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f962f85d-d19b-4e97-9c73-3ef0e2bd0f57 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:00.156745 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7f5495-9805-49bd-81e3-58a3b999b30d None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:00.157035 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7f5495-9805-49bd-81e3-58a3b999b30d None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:00.157308 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7f5495-9805-49bd-81e3-58a3b999b30d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:00.176201 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-4a7f5495-9805-49bd-81e3-58a3b999b30d None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:18:00.177393 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-4a7f5495-9805-49bd-81e3-58a3b999b30d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:18:00.197468 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38147636-607d-4555-ab60-c892af8f51ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:00.198078 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38147636-607d-4555-ab60-c892af8f51ef None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:00.198390 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38147636-607d-4555-ab60-c892af8f51ef None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:00.198661 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38147636-607d-4555-ab60-c892af8f51ef None admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:00.199324 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38147636-607d-4555-ab60-c892af8f51ef None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:18:00.199634 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38147636-607d-4555-ab60-c892af8f51ef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:18:00.200995 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38147636-607d-4555-ab60-c892af8f51ef None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:18:00.210865 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 391/789] 10.222.0.18 () {56 vars in 1115 bytes} [Wed Nov 2 07:18:00 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:18:00.216869 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38147636-607d-4555-ab60-c892af8f51ef None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:00.253961 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 399/790] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Nov 2 07:18:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Nov 02 07:18:00.330090 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38147636-607d-4555-ab60-c892af8f51ef None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:00.362302 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:18:00.370300 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6ca318-c647-4aa7-bcde-b16d8d0b70b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:00.370955 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6ca318-c647-4aa7-bcde-b16d8d0b70b7 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:00.371236 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6ca318-c647-4aa7-bcde-b16d8d0b70b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:00.371507 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6ca318-c647-4aa7-bcde-b16d8d0b70b7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:00.382552 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6ca318-c647-4aa7-bcde-b16d8d0b70b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:18:00.384108 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6ca318-c647-4aa7-bcde-b16d8d0b70b7 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:18:00.398724 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-4e6ca318-c647-4aa7-bcde-b16d8d0b70b7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:18:00.398724 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:18:00.398724 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:18:00.398724 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:18:00.398724 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:18:00.398724 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:18:00.398724 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:18:00.398724 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:18:00.398724 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:18:00.398724 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:18:00.398724 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:18:00.398724 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:18:00.398724 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:18:00.398724 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:18:00.398724 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:18:00.398724 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:18:00.399662 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:18:00.399662 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:18:00.399662 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:18:00.399662 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:18:00.399662 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:18:00.399662 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:18:00.399662 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:18:00.399662 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Nov 02 07:18:00.399662 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:18:00.399662 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:18:00.399662 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:18:00.399662 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:18:00.399662 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:18:00.399662 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:18:00.399662 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:18:00.399662 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:18:00.399662 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:18:00.405325 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:18:00.405325 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:18:00.405325 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Nov 02 07:18:00.405325 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 392/791] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Nov 2 07:18:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 07:18:00.407435 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7f5495-9805-49bd-81e3-58a3b999b30d None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:00.411030 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:18:00.508869 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7f5495-9805-49bd-81e3-58a3b999b30d None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:00.528861 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6ca318-c647-4aa7-bcde-b16d8d0b70b7 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:00.546798 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b48db17-7f9e-4a59-bf65-b0d5d363d1e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:00.547688 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b48db17-7f9e-4a59-bf65-b0d5d363d1e1 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:00.547968 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b48db17-7f9e-4a59-bf65-b0d5d363d1e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:00.548242 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b48db17-7f9e-4a59-bf65-b0d5d363d1e1 None admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:00.549048 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b48db17-7f9e-4a59-bf65-b0d5d363d1e1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:18:00.549644 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b48db17-7f9e-4a59-bf65-b0d5d363d1e1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:18:00.551630 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b48db17-7f9e-4a59-bf65-b0d5d363d1e1 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:18:00.565307 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 400/792] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Nov 2 07:18:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:18:00.572195 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b48db17-7f9e-4a59-bf65-b0d5d363d1e1 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:00.638079 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:18:00.643701 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6ca318-c647-4aa7-bcde-b16d8d0b70b7 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:00.685973 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64a874c-135e-4955-ac45-5152bf7bc4c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:00.686555 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64a874c-135e-4955-ac45-5152bf7bc4c8 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:00.686838 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64a874c-135e-4955-ac45-5152bf7bc4c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:00.687111 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64a874c-135e-4955-ac45-5152bf7bc4c8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:00.687735 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b48db17-7f9e-4a59-bf65-b0d5d363d1e1 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:00.688213 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64a874c-135e-4955-ac45-5152bf7bc4c8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:18:00.689645 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64a874c-135e-4955-ac45-5152bf7bc4c8 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:18:00.697336 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 393/793] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Nov 2 07:18:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:18:00.698008 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64a874c-135e-4955-ac45-5152bf7bc4c8 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:00.724680 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ab82da-0614-4782-a198-0a05f4e65efa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:00.725322 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ab82da-0614-4782-a198-0a05f4e65efa None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:00.725605 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ab82da-0614-4782-a198-0a05f4e65efa None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:00.725878 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ab82da-0614-4782-a198-0a05f4e65efa None admin] PATH_INFO: `/v3/users/nova` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:00.734280 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ab82da-0614-4782-a198-0a05f4e65efa None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:18:00.735763 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ab82da-0614-4782-a198-0a05f4e65efa None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:18:00.748086 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-44ab82da-0614-4782-a198-0a05f4e65efa None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Nov 02 07:18:00.748086 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:18:00.748086 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:18:00.748086 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:18:00.748086 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:18:00.748086 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:18:00.748086 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:18:00.748086 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:18:00.748086 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:18:00.748086 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:18:00.748086 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:18:00.748086 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:18:00.748086 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Nov 02 07:18:00.748086 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_user(user_id) Nov 02 07:18:00.748086 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Nov 02 07:18:00.748086 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Nov 02 07:18:00.749295 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:18:00.749295 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:18:00.749295 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Nov 02 07:18:00.749295 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:18:00.749295 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Nov 02 07:18:00.749295 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:18:00.749295 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:18:00.749295 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:18:00.749295 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:18:00.749295 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:18:00.749295 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:18:00.749295 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:18:00.749295 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:18:00.749295 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:18:00.749295 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:18:00.749295 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:18:00.749295 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:18:00.749295 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:18:00.750218 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:18:00.750218 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:18:00.750218 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Nov 02 07:18:00.750218 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Nov 02 07:18:00.750218 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Nov 02 07:18:00.750218 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Nov 02 07:18:00.750218 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Nov 02 07:18:00.750218 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Nov 02 07:18:00.750218 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Nov 02 07:18:00.750218 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:18:00.750218 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 401/794] 10.222.0.18 () {56 vars in 1093 bytes} [Wed Nov 2 07:18:00 2022] GET /identity/v3/users/nova => generated 82 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 07:18:00.752580 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:18:00.755140 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ab82da-0614-4782-a198-0a05f4e65efa None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:00.807909 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64a874c-135e-4955-ac45-5152bf7bc4c8 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:00.842795 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf1a16f-0f5c-46a2-93bd-6ae3e6dda7a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:00.843357 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf1a16f-0f5c-46a2-93bd-6ae3e6dda7a1 None admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:00.843638 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf1a16f-0f5c-46a2-93bd-6ae3e6dda7a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:00.843912 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf1a16f-0f5c-46a2-93bd-6ae3e6dda7a1 None admin] PATH_INFO: `/v3/projects` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:00.844550 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf1a16f-0f5c-46a2-93bd-6ae3e6dda7a1 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:18:00.845992 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf1a16f-0f5c-46a2-93bd-6ae3e6dda7a1 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:18:00.858294 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ab82da-0614-4782-a198-0a05f4e65efa None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:00.874919 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 394/795] 10.222.0.18 () {60 vars in 1141 bytes} [Wed Nov 2 07:18:00 2022] POST /identity/v3/projects => generated 311 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:18:00.886537 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:18:00.900533 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efc7901-450d-4fd4-b7dc-f601c5f7143a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:00.901223 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efc7901-450d-4fd4-b7dc-f601c5f7143a None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:00.901504 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efc7901-450d-4fd4-b7dc-f601c5f7143a None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:00.901826 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efc7901-450d-4fd4-b7dc-f601c5f7143a None admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:00.902618 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efc7901-450d-4fd4-b7dc-f601c5f7143a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:18:00.902928 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efc7901-450d-4fd4-b7dc-f601c5f7143a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:18:00.904475 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efc7901-450d-4fd4-b7dc-f601c5f7143a None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:18:00.925728 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 402/796] 10.222.0.18 () {56 vars in 1133 bytes} [Wed Nov 2 07:18:00 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:18:00.932129 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf1a16f-0f5c-46a2-93bd-6ae3e6dda7a1 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:01.042157 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf1a16f-0f5c-46a2-93bd-6ae3e6dda7a1 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:01.043840 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:18:01.084362 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c976fd-2698-446c-82f4-8e946cf1b6fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:01.085681 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c976fd-2698-446c-82f4-8e946cf1b6fb None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:01.085968 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c976fd-2698-446c-82f4-8e946cf1b6fb None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:01.086238 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c976fd-2698-446c-82f4-8e946cf1b6fb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:01.097055 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c976fd-2698-446c-82f4-8e946cf1b6fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:18:01.099280 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c976fd-2698-446c-82f4-8e946cf1b6fb None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:18:01.115094 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-06c976fd-2698-446c-82f4-8e946cf1b6fb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:18:01.115094 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:18:01.115094 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:18:01.115094 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:18:01.115094 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:18:01.115094 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:18:01.115094 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:18:01.115094 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:18:01.115094 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:18:01.115094 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:18:01.115094 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:18:01.115094 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:18:01.115094 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:18:01.115094 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:18:01.115094 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:18:01.115094 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:18:01.116072 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:18:01.116072 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:18:01.116072 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:18:01.116072 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:18:01.116072 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:18:01.116072 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:18:01.116072 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:18:01.116072 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Nov 02 07:18:01.116072 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:18:01.116072 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:18:01.116072 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:18:01.116072 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:18:01.116072 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:18:01.116072 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:18:01.116072 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:18:01.116072 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:18:01.116072 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:18:01.117531 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:18:01.117531 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:18:01.117531 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Nov 02 07:18:01.117531 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 395/797] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Nov 2 07:18:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:18:01.121936 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efc7901-450d-4fd4-b7dc-f601c5f7143a None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:01.228371 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efc7901-450d-4fd4-b7dc-f601c5f7143a None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:01.262456 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbc293c-d973-43f3-9c3f-96ee60f66cf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:01.263057 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbc293c-d973-43f3-9c3f-96ee60f66cf3 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:01.263335 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbc293c-d973-43f3-9c3f-96ee60f66cf3 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:01.263608 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbc293c-d973-43f3-9c3f-96ee60f66cf3 None admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:01.264239 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbc293c-d973-43f3-9c3f-96ee60f66cf3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:18:01.264534 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbc293c-d973-43f3-9c3f-96ee60f66cf3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:18:01.265903 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbc293c-d973-43f3-9c3f-96ee60f66cf3 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:18:01.277281 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 403/798] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Nov 2 07:18:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:18:01.283205 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c976fd-2698-446c-82f4-8e946cf1b6fb None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:01.387174 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c976fd-2698-446c-82f4-8e946cf1b6fb None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:01.421625 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da70961-e2b4-4a97-864b-6a8153c621b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:01.422253 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da70961-e2b4-4a97-864b-6a8153c621b0 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:01.422568 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da70961-e2b4-4a97-864b-6a8153c621b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:01.422838 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da70961-e2b4-4a97-864b-6a8153c621b0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:01.430180 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da70961-e2b4-4a97-864b-6a8153c621b0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:18:01.431648 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da70961-e2b4-4a97-864b-6a8153c621b0 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:18:01.442938 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-3da70961-e2b4-4a97-864b-6a8153c621b0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 07:18:01.442938 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:18:01.442938 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:18:01.442938 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:18:01.442938 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:18:01.442938 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:18:01.442938 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:18:01.442938 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:18:01.442938 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:18:01.442938 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:18:01.442938 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:18:01.442938 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:18:01.442938 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Nov 02 07:18:01.442938 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_project(project_id) Nov 02 07:18:01.442938 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Nov 02 07:18:01.443907 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Nov 02 07:18:01.443907 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:18:01.443907 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:18:01.443907 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:18:01.443907 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:18:01.443907 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:18:01.443907 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:18:01.443907 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:18:01.443907 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Nov 02 07:18:01.443907 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:18:01.443907 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:18:01.443907 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:18:01.443907 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:18:01.443907 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:18:01.443907 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:18:01.443907 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:18:01.443907 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:18:01.449936 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Nov 02 07:18:01.449936 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Nov 02 07:18:01.449936 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Nov 02 07:18:01.449936 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Nov 02 07:18:01.449936 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Nov 02 07:18:01.449936 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Nov 02 07:18:01.449936 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 07:18:01.449936 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Nov 02 07:18:01.449936 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 396/799] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Nov 2 07:18:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 07:18:01.452335 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbc293c-d973-43f3-9c3f-96ee60f66cf3 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:01.498616 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:18:01.557246 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbc293c-d973-43f3-9c3f-96ee60f66cf3 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:01.568457 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d857afc0-21dc-41aa-ac8a-8b85da52c812 None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:01.568720 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d857afc0-21dc-41aa-ac8a-8b85da52c812 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:01.568995 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d857afc0-21dc-41aa-ac8a-8b85da52c812 None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:01.574399 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 397/800] 10.222.0.18 () {54 vars in 927 bytes} [Wed Nov 2 07:18:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Nov 02 07:18:01.575442 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf56c23f-7007-4f76-9343-4e83edaaed89 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:01.575709 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf56c23f-7007-4f76-9343-4e83edaaed89 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:01.575978 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf56c23f-7007-4f76-9343-4e83edaaed89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:01.594675 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-cf56c23f-7007-4f76-9343-4e83edaaed89 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:18:01.594985 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80db0216-36f4-40de-a0eb-a4617d92f248 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:01.595484 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80db0216-36f4-40de-a0eb-a4617d92f248 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:01.595756 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80db0216-36f4-40de-a0eb-a4617d92f248 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:01.595874 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-cf56c23f-7007-4f76-9343-4e83edaaed89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:18:01.596036 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80db0216-36f4-40de-a0eb-a4617d92f248 None admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:01.596690 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80db0216-36f4-40de-a0eb-a4617d92f248 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:18:01.596988 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80db0216-36f4-40de-a0eb-a4617d92f248 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:18:01.598522 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80db0216-36f4-40de-a0eb-a4617d92f248 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:18:01.606845 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:18:01.611144 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 404/801] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Nov 2 07:18:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:18:01.618214 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80db0216-36f4-40de-a0eb-a4617d92f248 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:01.675146 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 398/802] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Nov 2 07:18:01 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Nov 02 07:18:01.726938 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:18:01.730497 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80db0216-36f4-40de-a0eb-a4617d92f248 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:01.765551 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0900e5-5cbd-4b0e-8330-af97765aae89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:01.766200 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0900e5-5cbd-4b0e-8330-af97765aae89 None admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:01.766502 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0900e5-5cbd-4b0e-8330-af97765aae89 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:01.766770 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0900e5-5cbd-4b0e-8330-af97765aae89 None admin] PATH_INFO: `/v3/projects/af04daa468b64160a6176dcab29502a7/users/69e48cba3f804cf08b2477f01ae95179/roles/26344c06182042078907487e8d1a0d9f` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:01.768888 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0900e5-5cbd-4b0e-8330-af97765aae89 None admin] RBAC: Authorizing `identity:create_grant(project_id=af04daa468b64160a6176dcab29502a7, user_id=69e48cba3f804cf08b2477f01ae95179, role_id=26344c06182042078907487e8d1a0d9f)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:18:01.770979 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0900e5-5cbd-4b0e-8330-af97765aae89 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:18:01.784295 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 405/803] 10.222.0.18 () {58 vars in 1439 bytes} [Wed Nov 2 07:18:01 2022] PUT /identity/v3/projects/af04daa468b64160a6176dcab29502a7/users/69e48cba3f804cf08b2477f01ae95179/roles/26344c06182042078907487e8d1a0d9f => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 07:18:01.946672 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf56c23f-7007-4f76-9343-4e83edaaed89 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:02.048062 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf56c23f-7007-4f76-9343-4e83edaaed89 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:02.082482 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29d7f61-2b91-45c3-ab05-e55427aa24cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:02.083416 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29d7f61-2b91-45c3-ab05-e55427aa24cb None admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:02.083908 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29d7f61-2b91-45c3-ab05-e55427aa24cb None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:02.084456 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29d7f61-2b91-45c3-ab05-e55427aa24cb None admin] PATH_INFO: `/v3/domains` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:02.085745 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29d7f61-2b91-45c3-ab05-e55427aa24cb None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:18:02.087361 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29d7f61-2b91-45c3-ab05-e55427aa24cb None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:18:02.095094 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.sql.core [None req-b29d7f61-2b91-45c3-ab05-e55427aa24cb None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Nov 02 07:18:02.095094 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Nov 02 07:18:02.095094 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [parameters: {'id': '37dad50a1697479ca6bf11c9f7953ccd', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Nov 02 07:18:02.095094 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81501) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Nov 02 07:18:02.104716 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-b29d7f61-2b91-45c3-ab05-e55427aa24cb None admin] Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Nov 02 07:18:02.104716 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:18:02.104716 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Nov 02 07:18:02.104716 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application self.dialect.do_execute( Nov 02 07:18:02.104716 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Nov 02 07:18:02.104716 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Nov 02 07:18:02.104716 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Nov 02 07:18:02.104716 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application result = self._query(query) Nov 02 07:18:02.104716 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Nov 02 07:18:02.104716 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application conn.query(q) Nov 02 07:18:02.104716 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Nov 02 07:18:02.104716 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Nov 02 07:18:02.104716 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Nov 02 07:18:02.104716 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application result.read() Nov 02 07:18:02.105563 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Nov 02 07:18:02.105563 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Nov 02 07:18:02.105563 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Nov 02 07:18:02.105563 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application packet.raise_for_error() Nov 02 07:18:02.105563 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Nov 02 07:18:02.105563 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Nov 02 07:18:02.105563 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Nov 02 07:18:02.105563 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Nov 02 07:18:02.105563 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Nov 02 07:18:02.105563 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Nov 02 07:18:02.105563 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Nov 02 07:18:02.105563 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Nov 02 07:18:02.105563 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:18:02.105563 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Nov 02 07:18:02.105563 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return method(*args, **kwargs) Nov 02 07:18:02.105563 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Nov 02 07:18:02.105563 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return project_ref.to_dict() Nov 02 07:18:02.105563 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Nov 02 07:18:02.107003 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application next(self.gen) Nov 02 07:18:02.107003 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1068, in _transaction_scope Nov 02 07:18:02.107003 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application yield resource Nov 02 07:18:02.107003 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Nov 02 07:18:02.107003 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application next(self.gen) Nov 02 07:18:02.107003 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 670, in _session Nov 02 07:18:02.107003 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application self.session.rollback() Nov 02 07:18:02.107003 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Nov 02 07:18:02.107003 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application self.force_reraise() Nov 02 07:18:02.107003 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Nov 02 07:18:02.107003 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise self.value Nov 02 07:18:02.107003 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 667, in _session Nov 02 07:18:02.107003 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Nov 02 07:18:02.107003 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 695, in _end_session_transaction Nov 02 07:18:02.107003 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application session.commit() Nov 02 07:18:02.107003 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Nov 02 07:18:02.107003 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Nov 02 07:18:02.107863 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Nov 02 07:18:02.107863 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application self._prepare_impl() Nov 02 07:18:02.107863 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Nov 02 07:18:02.107863 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application self.session.flush() Nov 02 07:18:02.107863 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Nov 02 07:18:02.107863 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application self._flush(objects) Nov 02 07:18:02.107863 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Nov 02 07:18:02.107863 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Nov 02 07:18:02.107863 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Nov 02 07:18:02.107863 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application compat.raise_( Nov 02 07:18:02.107863 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Nov 02 07:18:02.107863 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception Nov 02 07:18:02.107863 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Nov 02 07:18:02.107863 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application flush_context.execute() Nov 02 07:18:02.107863 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Nov 02 07:18:02.107863 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rec.execute(self) Nov 02 07:18:02.107863 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Nov 02 07:18:02.108728 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Nov 02 07:18:02.108728 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Nov 02 07:18:02.108728 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application _emit_insert_statements( Nov 02 07:18:02.108728 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Nov 02 07:18:02.108728 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application c = connection._execute_20( Nov 02 07:18:02.108728 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Nov 02 07:18:02.108728 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Nov 02 07:18:02.108728 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Nov 02 07:18:02.108728 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Nov 02 07:18:02.108728 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Nov 02 07:18:02.108728 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application ret = self._execute_context( Nov 02 07:18:02.108728 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Nov 02 07:18:02.108728 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Nov 02 07:18:02.108728 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Nov 02 07:18:02.108728 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Nov 02 07:18:02.109486 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Nov 02 07:18:02.109486 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception Nov 02 07:18:02.109486 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Nov 02 07:18:02.109486 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application self.dialect.do_execute( Nov 02 07:18:02.109486 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Nov 02 07:18:02.109486 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Nov 02 07:18:02.109486 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Nov 02 07:18:02.109486 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application result = self._query(query) Nov 02 07:18:02.109486 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Nov 02 07:18:02.109486 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application conn.query(q) Nov 02 07:18:02.109486 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Nov 02 07:18:02.109486 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Nov 02 07:18:02.109486 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Nov 02 07:18:02.109486 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application result.read() Nov 02 07:18:02.109486 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Nov 02 07:18:02.109486 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Nov 02 07:18:02.109486 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Nov 02 07:18:02.110371 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application packet.raise_for_error() Nov 02 07:18:02.110371 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Nov 02 07:18:02.110371 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Nov 02 07:18:02.110371 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Nov 02 07:18:02.110371 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Nov 02 07:18:02.110371 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Nov 02 07:18:02.110371 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Nov 02 07:18:02.110371 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [parameters: {'id': '37dad50a1697479ca6bf11c9f7953ccd', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Nov 02 07:18:02.110371 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Nov 02 07:18:02.110371 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Nov 02 07:18:02.110371 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 07:18:02.110371 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Nov 02 07:18:02.110371 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:18:02.110371 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Nov 02 07:18:02.110371 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Nov 02 07:18:02.112657 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Nov 02 07:18:02.112657 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Nov 02 07:18:02.112657 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name Default. Nov 02 07:18:02.112657 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Nov 02 07:18:02.112657 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 07:18:02.112657 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Nov 02 07:18:02.112657 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:18:02.112657 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:18:02.112657 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:18:02.112657 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:18:02.112657 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:18:02.112657 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:18:02.112657 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:18:02.112657 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:18:02.112657 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:18:02.112657 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:18:02.112657 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:18:02.115030 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 128, in post Nov 02 07:18:02.115030 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_domain( Nov 02 07:18:02.115030 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:18:02.115030 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:18:02.115030 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 765, in create_domain Nov 02 07:18:02.115030 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application is_domain_project = self.create_project( Nov 02 07:18:02.115030 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:18:02.115030 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:18:02.115030 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Nov 02 07:18:02.115030 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.Conflict( Nov 02 07:18:02.115030 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Nov 02 07:18:02.115030 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Nov 02 07:18:02.115030 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 399/804] 10.222.0.18 () {60 vars in 1137 bytes} [Wed Nov 2 07:18:01 2022] POST /identity/v3/domains => generated 190 bytes in 161 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:18:02.120189 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0900e5-5cbd-4b0e-8330-af97765aae89 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:02.228142 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0900e5-5cbd-4b0e-8330-af97765aae89 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:02.263116 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9ba82c-08fe-4b34-b379-ec249e91f46f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:02.263887 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9ba82c-08fe-4b34-b379-ec249e91f46f None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:02.264287 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9ba82c-08fe-4b34-b379-ec249e91f46f None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:02.264650 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9ba82c-08fe-4b34-b379-ec249e91f46f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:02.275644 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9ba82c-08fe-4b34-b379-ec249e91f46f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:18:02.277165 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9ba82c-08fe-4b34-b379-ec249e91f46f None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:18:02.292378 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-3c9ba82c-08fe-4b34-b379-ec249e91f46f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:18:02.292378 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:18:02.292378 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:18:02.292378 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:18:02.292378 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:18:02.292378 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:18:02.292378 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:18:02.292378 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:18:02.292378 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:18:02.292378 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:18:02.292378 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:18:02.292378 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:18:02.292378 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:18:02.292378 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:18:02.292378 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:18:02.292378 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:18:02.293549 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:18:02.293549 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:18:02.293549 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:18:02.293549 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:18:02.293549 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:18:02.293549 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:18:02.293549 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:18:02.293549 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:18:02.293549 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:18:02.293549 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:18:02.293549 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:18:02.293549 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:18:02.293549 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:18:02.293549 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:18:02.293549 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:18:02.293549 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:18:02.293549 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:18:02.294691 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:18:02.294691 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:18:02.294691 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:18:02.295873 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 406/805] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Nov 2 07:18:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:18:02.301635 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29d7f61-2b91-45c3-ab05-e55427aa24cb None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:02.312568 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:18:02.406166 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29d7f61-2b91-45c3-ab05-e55427aa24cb None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:02.440308 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd08102e-630b-4633-837b-044a751dbceb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:02.441090 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd08102e-630b-4633-837b-044a751dbceb None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:02.441496 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd08102e-630b-4633-837b-044a751dbceb None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:02.441947 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd08102e-630b-4633-837b-044a751dbceb None admin] PATH_INFO: `/v3/domains` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:02.442737 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd08102e-630b-4633-837b-044a751dbceb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:18:02.443142 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd08102e-630b-4633-837b-044a751dbceb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:18:02.444626 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd08102e-630b-4633-837b-044a751dbceb None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:18:02.462389 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 400/806] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Nov 2 07:18:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:18:03.111813 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcc1430-de18-425c-a3a5-6ee8a291ebdb None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:03.112092 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcc1430-de18-425c-a3a5-6ee8a291ebdb None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:03.112351 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcc1430-de18-425c-a3a5-6ee8a291ebdb None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:03.113905 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 407/807] 10.222.0.18 () {54 vars in 927 bytes} [Wed Nov 2 07:18:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:18:03.119122 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1d7e0d-b6fe-4f59-b0cd-8898f0cc62fb None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:03.119586 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1d7e0d-b6fe-4f59-b0cd-8898f0cc62fb None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:03.119833 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1d7e0d-b6fe-4f59-b0cd-8898f0cc62fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:03.137770 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-3a1d7e0d-b6fe-4f59-b0cd-8898f0cc62fb None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:18:03.138967 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-3a1d7e0d-b6fe-4f59-b0cd-8898f0cc62fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:18:03.201175 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:18:03.208491 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 401/808] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Nov 2 07:18:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 07:18:03.480969 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcc1430-de18-425c-a3a5-6ee8a291ebdb None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:03.496422 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:18:03.581040 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcc1430-de18-425c-a3a5-6ee8a291ebdb None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:03.615051 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fae4476-8ad5-4738-998c-0b1e8f6052cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:03.615671 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fae4476-8ad5-4738-998c-0b1e8f6052cd None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:03.615946 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fae4476-8ad5-4738-998c-0b1e8f6052cd None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:03.616215 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fae4476-8ad5-4738-998c-0b1e8f6052cd None admin] PATH_INFO: `/v3/domains/default` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:03.617297 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fae4476-8ad5-4738-998c-0b1e8f6052cd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:18:03.618743 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fae4476-8ad5-4738-998c-0b1e8f6052cd None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:18:03.620429 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 408/809] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Nov 2 07:18:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:18:03.625947 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1d7e0d-b6fe-4f59-b0cd-8898f0cc62fb None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:03.724999 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1d7e0d-b6fe-4f59-b0cd-8898f0cc62fb None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:03.757377 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0fe543-b764-4074-a5cc-6a20e86e3876 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:03.757976 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0fe543-b764-4074-a5cc-6a20e86e3876 None admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:03.758255 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0fe543-b764-4074-a5cc-6a20e86e3876 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:03.758551 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0fe543-b764-4074-a5cc-6a20e86e3876 None admin] PATH_INFO: `/v3/users` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:03.759186 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0fe543-b764-4074-a5cc-6a20e86e3876 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:18:03.760609 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0fe543-b764-4074-a5cc-6a20e86e3876 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:18:03.794392 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 402/810] 10.222.0.18 () {60 vars in 1132 bytes} [Wed Nov 2 07:18:03 2022] POST /identity/v3/users => generated 285 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:18:03.963193 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cd8a64-9241-4d06-bf3d-a1e60f05562a None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:03.963473 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cd8a64-9241-4d06-bf3d-a1e60f05562a None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:03.963739 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cd8a64-9241-4d06-bf3d-a1e60f05562a None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:03.965068 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 409/811] 10.222.0.18 () {54 vars in 927 bytes} [Wed Nov 2 07:18:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:18:03.970178 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a852346-1f28-4771-8149-03461863e903 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:03.970521 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a852346-1f28-4771-8149-03461863e903 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:03.970801 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a852346-1f28-4771-8149-03461863e903 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:03.989932 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-9a852346-1f28-4771-8149-03461863e903 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:18:03.991164 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-9a852346-1f28-4771-8149-03461863e903 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:18:04.040015 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:18:04.058265 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 403/812] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Nov 2 07:18:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 07:18:04.326924 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cd8a64-9241-4d06-bf3d-a1e60f05562a None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:04.409675 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:18:04.431358 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cd8a64-9241-4d06-bf3d-a1e60f05562a None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:04.465139 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a118ee71-a28e-4252-bfb2-e98571cd0cc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:04.465754 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a118ee71-a28e-4252-bfb2-e98571cd0cc3 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:04.466030 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a118ee71-a28e-4252-bfb2-e98571cd0cc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:04.466294 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a118ee71-a28e-4252-bfb2-e98571cd0cc3 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:04.477383 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a118ee71-a28e-4252-bfb2-e98571cd0cc3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:18:04.482346 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-a118ee71-a28e-4252-bfb2-e98571cd0cc3 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Nov 02 07:18:04.482346 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:18:04.482346 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:18:04.482346 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:18:04.482346 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:18:04.482346 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:18:04.482346 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:18:04.482346 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:18:04.482346 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:18:04.482346 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:18:04.482346 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:18:04.482346 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:18:04.482346 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:18:04.482346 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:18:04.482346 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:18:04.482346 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:18:04.483871 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:18:04.483871 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:18:04.483871 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:18:04.483871 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:18:04.483871 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:18:04.483871 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:18:04.483871 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:18:04.483871 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:18:04.483871 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:18:04.483871 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:18:04.483871 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:18:04.483871 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:18:04.483871 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:18:04.483871 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:18:04.483871 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:18:04.483871 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:18:04.484761 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:18:04.484761 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:18:04.484761 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:18:04.484761 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:18:04.484761 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:18:04.484761 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:18:04.484761 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:18:04.484761 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:18:04.484761 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:18:04.484761 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:18:04.484761 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Nov 02 07:18:04.484761 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:18:04.484761 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 410/813] 10.222.0.18 () {56 vars in 1120 bytes} [Wed Nov 2 07:18:04 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:18:04.489710 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a852346-1f28-4771-8149-03461863e903 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:04.585107 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a852346-1f28-4771-8149-03461863e903 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:04.617665 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0f21b3-0639-4f42-bca5-ecb0ac60b5e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:04.618266 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0f21b3-0639-4f42-bca5-ecb0ac60b5e8 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:04.618577 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0f21b3-0639-4f42-bca5-ecb0ac60b5e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:04.618846 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0f21b3-0639-4f42-bca5-ecb0ac60b5e8 None admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:04.619490 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0f21b3-0639-4f42-bca5-ecb0ac60b5e8 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:18:04.619785 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0f21b3-0639-4f42-bca5-ecb0ac60b5e8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:18:04.621131 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0f21b3-0639-4f42-bca5-ecb0ac60b5e8 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:18:04.628723 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 404/814] 10.222.0.18 () {56 vars in 1115 bytes} [Wed Nov 2 07:18:04 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:18:04.634483 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a118ee71-a28e-4252-bfb2-e98571cd0cc3 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:04.733695 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a118ee71-a28e-4252-bfb2-e98571cd0cc3 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:04.765512 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9542532-c201-4cfb-9226-eeda96ec0fb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:04.766126 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9542532-c201-4cfb-9226-eeda96ec0fb5 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:04.766426 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9542532-c201-4cfb-9226-eeda96ec0fb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:04.766694 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9542532-c201-4cfb-9226-eeda96ec0fb5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:04.775609 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:18:04.778106 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9542532-c201-4cfb-9226-eeda96ec0fb5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:18:04.779960 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9542532-c201-4cfb-9226-eeda96ec0fb5 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:18:04.794072 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-c9542532-c201-4cfb-9226-eeda96ec0fb5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:18:04.794072 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:18:04.794072 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:18:04.794072 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:18:04.794072 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:18:04.794072 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:18:04.794072 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:18:04.794072 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:18:04.794072 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:18:04.794072 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:18:04.794072 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:18:04.794072 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:18:04.794072 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:18:04.794072 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:18:04.794072 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:18:04.794072 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:18:04.795089 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:18:04.795089 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:18:04.795089 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:18:04.795089 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:18:04.795089 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:18:04.795089 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:18:04.795089 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:18:04.795089 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:18:04.795089 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:18:04.795089 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:18:04.795089 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:18:04.795089 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:18:04.795089 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:18:04.795089 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:18:04.795089 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:18:04.795089 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:18:04.795089 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:18:04.796301 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:18:04.796301 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:18:04.796301 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:18:04.796301 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 411/815] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Nov 2 07:18:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:18:04.801375 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0f21b3-0639-4f42-bca5-ecb0ac60b5e8 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:04.846608 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:18:04.898341 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0f21b3-0639-4f42-bca5-ecb0ac60b5e8 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:04.931549 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d39352-fbaf-4a23-a648-b8fe2c24e67c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:04.932145 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d39352-fbaf-4a23-a648-b8fe2c24e67c None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:04.932420 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d39352-fbaf-4a23-a648-b8fe2c24e67c None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:04.932693 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d39352-fbaf-4a23-a648-b8fe2c24e67c None admin] PATH_INFO: `/v3/domains` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:04.933313 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d39352-fbaf-4a23-a648-b8fe2c24e67c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:18:04.933608 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d39352-fbaf-4a23-a648-b8fe2c24e67c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:18:04.934991 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d39352-fbaf-4a23-a648-b8fe2c24e67c None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:18:04.946328 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 405/816] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Nov 2 07:18:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:18:04.952109 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9542532-c201-4cfb-9226-eeda96ec0fb5 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:05.051203 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9542532-c201-4cfb-9226-eeda96ec0fb5 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:05.083053 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4f0a6c-5363-43d6-b7bb-4c513c32fa88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:05.083669 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4f0a6c-5363-43d6-b7bb-4c513c32fa88 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:05.083943 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4f0a6c-5363-43d6-b7bb-4c513c32fa88 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:05.084204 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4f0a6c-5363-43d6-b7bb-4c513c32fa88 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:05.091312 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4f0a6c-5363-43d6-b7bb-4c513c32fa88 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:18:05.092731 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4f0a6c-5363-43d6-b7bb-4c513c32fa88 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:18:05.104355 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-be4f0a6c-5363-43d6-b7bb-4c513c32fa88 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Nov 02 07:18:05.104355 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:18:05.104355 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:18:05.104355 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:18:05.104355 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:18:05.104355 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:18:05.104355 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:18:05.104355 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:18:05.104355 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:18:05.104355 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:18:05.104355 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:18:05.104355 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:18:05.104355 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Nov 02 07:18:05.104355 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_user(user_id) Nov 02 07:18:05.104355 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Nov 02 07:18:05.104355 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Nov 02 07:18:05.105488 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:18:05.105488 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:18:05.105488 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Nov 02 07:18:05.105488 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:18:05.105488 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Nov 02 07:18:05.105488 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:18:05.105488 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:18:05.105488 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:18:05.105488 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:18:05.105488 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:18:05.105488 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:18:05.105488 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:18:05.105488 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:18:05.105488 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:18:05.105488 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:18:05.105488 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:18:05.105488 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:18:05.105488 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:18:05.106712 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:18:05.106712 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:18:05.106712 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Nov 02 07:18:05.106712 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Nov 02 07:18:05.106712 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Nov 02 07:18:05.106712 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Nov 02 07:18:05.106712 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Nov 02 07:18:05.106712 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Nov 02 07:18:05.106712 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Nov 02 07:18:05.106712 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:18:05.106712 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 412/817] 10.222.0.18 () {56 vars in 1093 bytes} [Wed Nov 2 07:18:04 2022] GET /identity/v3/users/nova => generated 82 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:18:05.111131 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d39352-fbaf-4a23-a648-b8fe2c24e67c None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:05.209158 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d39352-fbaf-4a23-a648-b8fe2c24e67c None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:05.216029 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:18:05.243029 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89996015-7f14-45aa-b0e6-c915240d5884 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:05.243626 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89996015-7f14-45aa-b0e6-c915240d5884 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:05.243897 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89996015-7f14-45aa-b0e6-c915240d5884 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:05.244159 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89996015-7f14-45aa-b0e6-c915240d5884 None admin] PATH_INFO: `/v3/users` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:05.244864 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89996015-7f14-45aa-b0e6-c915240d5884 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:18:05.245162 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89996015-7f14-45aa-b0e6-c915240d5884 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:18:05.246593 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89996015-7f14-45aa-b0e6-c915240d5884 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:18:05.262840 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 406/818] 10.222.0.18 () {56 vars in 1133 bytes} [Wed Nov 2 07:18:05 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:18:05.268512 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4f0a6c-5363-43d6-b7bb-4c513c32fa88 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:05.318874 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:18:05.368605 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4f0a6c-5363-43d6-b7bb-4c513c32fa88 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:05.401915 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fb96be-069a-4740-b33c-3f4d04338202 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:05.402537 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fb96be-069a-4740-b33c-3f4d04338202 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:05.402813 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fb96be-069a-4740-b33c-3f4d04338202 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:05.403078 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fb96be-069a-4740-b33c-3f4d04338202 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:05.468495 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fb96be-069a-4740-b33c-3f4d04338202 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:18:05.469923 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fb96be-069a-4740-b33c-3f4d04338202 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:18:05.519763 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:18:05.567920 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 413/819] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Nov 2 07:18:05 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Nov 02 07:18:05.573580 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89996015-7f14-45aa-b0e6-c915240d5884 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:05.670526 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89996015-7f14-45aa-b0e6-c915240d5884 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:05.703698 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d48ad0-f8c8-45fe-aeac-2e226b83d568 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:05.704320 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d48ad0-f8c8-45fe-aeac-2e226b83d568 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:05.704598 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d48ad0-f8c8-45fe-aeac-2e226b83d568 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:05.704868 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d48ad0-f8c8-45fe-aeac-2e226b83d568 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:05.714781 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d48ad0-f8c8-45fe-aeac-2e226b83d568 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:18:05.716207 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d48ad0-f8c8-45fe-aeac-2e226b83d568 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:18:05.730937 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-12d48ad0-f8c8-45fe-aeac-2e226b83d568 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:18:05.730937 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:18:05.730937 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:18:05.730937 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:18:05.730937 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:18:05.730937 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:18:05.730937 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:18:05.730937 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:18:05.730937 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:18:05.730937 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:18:05.730937 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:18:05.730937 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:18:05.730937 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:18:05.730937 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:18:05.730937 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:18:05.730937 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:18:05.732043 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:18:05.732043 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:18:05.732043 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:18:05.732043 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:18:05.732043 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:18:05.732043 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:18:05.732043 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:18:05.732043 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Nov 02 07:18:05.732043 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:18:05.732043 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:18:05.732043 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:18:05.732043 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:18:05.732043 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:18:05.732043 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:18:05.732043 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:18:05.732043 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:18:05.732043 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:18:05.732935 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:18:05.732935 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:18:05.732935 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Nov 02 07:18:05.732935 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 407/820] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Nov 2 07:18:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:18:05.737724 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fb96be-069a-4740-b33c-3f4d04338202 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:05.832926 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fb96be-069a-4740-b33c-3f4d04338202 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:05.869282 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daab9e57-eea8-40e1-b326-9bd6b787b81b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:05.869904 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daab9e57-eea8-40e1-b326-9bd6b787b81b None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:05.870182 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daab9e57-eea8-40e1-b326-9bd6b787b81b None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:05.870483 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daab9e57-eea8-40e1-b326-9bd6b787b81b None admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:05.871145 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daab9e57-eea8-40e1-b326-9bd6b787b81b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:18:05.871444 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daab9e57-eea8-40e1-b326-9bd6b787b81b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:18:05.872827 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daab9e57-eea8-40e1-b326-9bd6b787b81b None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:18:05.885161 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 414/821] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Nov 2 07:18:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:18:05.890899 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d48ad0-f8c8-45fe-aeac-2e226b83d568 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:05.909142 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:18:05.968131 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56ab7fc-3c60-4a0f-be36-435a1f5a0220 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:05.968420 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56ab7fc-3c60-4a0f-be36-435a1f5a0220 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:05.968700 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56ab7fc-3c60-4a0f-be36-435a1f5a0220 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:05.970412 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 415/822] 10.222.0.18 () {54 vars in 927 bytes} [Wed Nov 2 07:18:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Nov 02 07:18:05.975923 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10476c31-5eeb-4946-8663-9e2360bd3317 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:05.976206 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10476c31-5eeb-4946-8663-9e2360bd3317 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:05.976476 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10476c31-5eeb-4946-8663-9e2360bd3317 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:05.992477 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d48ad0-f8c8-45fe-aeac-2e226b83d568 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:05.994437 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-10476c31-5eeb-4946-8663-9e2360bd3317 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:18:05.995636 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-10476c31-5eeb-4946-8663-9e2360bd3317 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:18:06.002770 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:18:06.026278 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e34ffa-562a-4688-9a32-90fd979a1778 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:06.026924 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e34ffa-562a-4688-9a32-90fd979a1778 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:06.027199 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e34ffa-562a-4688-9a32-90fd979a1778 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:06.027463 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e34ffa-562a-4688-9a32-90fd979a1778 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:06.033939 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e34ffa-562a-4688-9a32-90fd979a1778 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:18:06.036011 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e34ffa-562a-4688-9a32-90fd979a1778 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:18:06.047348 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-67e34ffa-562a-4688-9a32-90fd979a1778 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 07:18:06.047348 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:18:06.047348 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:18:06.047348 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:18:06.047348 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:18:06.047348 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:18:06.047348 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:18:06.047348 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:18:06.047348 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:18:06.047348 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:18:06.047348 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:18:06.047348 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:18:06.047348 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Nov 02 07:18:06.047348 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_project(project_id) Nov 02 07:18:06.047348 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Nov 02 07:18:06.048384 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Nov 02 07:18:06.048384 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:18:06.048384 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:18:06.048384 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:18:06.048384 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:18:06.048384 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:18:06.048384 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:18:06.048384 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:18:06.048384 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Nov 02 07:18:06.048384 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:18:06.048384 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:18:06.048384 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:18:06.048384 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:18:06.048384 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:18:06.048384 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:18:06.048384 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:18:06.048384 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:18:06.054225 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Nov 02 07:18:06.054225 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Nov 02 07:18:06.054225 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Nov 02 07:18:06.054225 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Nov 02 07:18:06.054225 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Nov 02 07:18:06.054225 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Nov 02 07:18:06.054225 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 07:18:06.054225 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Nov 02 07:18:06.054225 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 408/823] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Nov 2 07:18:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:18:06.054844 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e34ffa-562a-4688-9a32-90fd979a1778 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:06.068193 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 416/824] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Nov 2 07:18:05 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Nov 02 07:18:06.129837 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:18:06.152124 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e34ffa-562a-4688-9a32-90fd979a1778 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:06.185054 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7bd556-33fb-41b4-8cb2-7e8980fe5bbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:06.185596 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7bd556-33fb-41b4-8cb2-7e8980fe5bbc None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:06.185872 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7bd556-33fb-41b4-8cb2-7e8980fe5bbc None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:06.186140 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7bd556-33fb-41b4-8cb2-7e8980fe5bbc None admin] PATH_INFO: `/v3/projects` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:06.186846 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7bd556-33fb-41b4-8cb2-7e8980fe5bbc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:18:06.187151 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7bd556-33fb-41b4-8cb2-7e8980fe5bbc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:18:06.188592 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7bd556-33fb-41b4-8cb2-7e8980fe5bbc None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:18:06.315229 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 409/825] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Nov 2 07:18:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:18:06.321837 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10476c31-5eeb-4946-8663-9e2360bd3317 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:06.340996 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7bd556-33fb-41b4-8cb2-7e8980fe5bbc None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:06.424535 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10476c31-5eeb-4946-8663-9e2360bd3317 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:06.438950 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7bd556-33fb-41b4-8cb2-7e8980fe5bbc None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:06.461674 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1528eb-a36e-4bed-8e0c-c8f656415500 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:06.462304 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1528eb-a36e-4bed-8e0c-c8f656415500 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:06.462602 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1528eb-a36e-4bed-8e0c-c8f656415500 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:06.462874 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1528eb-a36e-4bed-8e0c-c8f656415500 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:06.463585 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1528eb-a36e-4bed-8e0c-c8f656415500 None admin] RBAC: Adding query filter params (role.id=26344c06182042078907487e8d1a0d9f, scope.project.id=af04daa468b64160a6176dcab29502a7, user.id=69e48cba3f804cf08b2477f01ae95179) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:18:06.463886 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1528eb-a36e-4bed-8e0c-c8f656415500 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:18:06.465304 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1528eb-a36e-4bed-8e0c-c8f656415500 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:18:06.471024 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 417/826] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Nov 2 07:18:06 2022] GET /identity/v3/role_assignments?user.id=69e48cba3f804cf08b2477f01ae95179&scope.project.id=af04daa468b64160a6176dcab29502a7&role.id=26344c06182042078907487e8d1a0d9f => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:18:06.476839 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c25b5ea-7c9c-492d-82b4-81ef47cddeba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:06.476839 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c25b5ea-7c9c-492d-82b4-81ef47cddeba None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:06.476839 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c25b5ea-7c9c-492d-82b4-81ef47cddeba None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:06.476839 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c25b5ea-7c9c-492d-82b4-81ef47cddeba None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:06.477165 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:18:06.486521 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c25b5ea-7c9c-492d-82b4-81ef47cddeba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:18:06.491450 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-0c25b5ea-7c9c-492d-82b4-81ef47cddeba None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 07:18:06.491450 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:18:06.491450 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:18:06.491450 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:18:06.491450 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:18:06.491450 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:18:06.491450 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:18:06.491450 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:18:06.491450 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:18:06.491450 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:18:06.491450 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:18:06.491450 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:18:06.491450 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:18:06.491450 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:18:06.491450 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:18:06.491450 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:18:06.492682 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:18:06.492682 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:18:06.492682 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:18:06.492682 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:18:06.492682 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:18:06.492682 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:18:06.492682 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:18:06.492682 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:18:06.492682 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:18:06.492682 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:18:06.492682 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:18:06.492682 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Nov 02 07:18:06.492682 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:18:06.492682 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:18:06.492682 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:18:06.492682 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:18:06.493543 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:18:06.493543 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:18:06.493543 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:18:06.493543 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:18:06.493543 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:18:06.493543 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:18:06.493543 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:18:06.493543 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:18:06.493543 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:18:06.493543 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:18:06.493543 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 07:18:06.493543 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Nov 02 07:18:06.493543 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 410/827] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Nov 2 07:18:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 07:18:06.498959 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1528eb-a36e-4bed-8e0c-c8f656415500 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:06.508950 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:18:06.604481 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1528eb-a36e-4bed-8e0c-c8f656415500 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:06.638811 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b570f19-d34a-4851-8fa3-b592d1ca262b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:06.639420 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b570f19-d34a-4851-8fa3-b592d1ca262b None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:06.639701 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b570f19-d34a-4851-8fa3-b592d1ca262b None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:06.639973 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b570f19-d34a-4851-8fa3-b592d1ca262b None admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:06.640635 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b570f19-d34a-4851-8fa3-b592d1ca262b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:18:06.640939 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b570f19-d34a-4851-8fa3-b592d1ca262b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:18:06.642292 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b570f19-d34a-4851-8fa3-b592d1ca262b None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:18:06.652610 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 418/828] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Nov 2 07:18:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:18:06.659242 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c25b5ea-7c9c-492d-82b4-81ef47cddeba None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:06.759528 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c25b5ea-7c9c-492d-82b4-81ef47cddeba None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:06.795392 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8463ee9-62d3-41b6-b5c9-6abe4c48a529 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:06.796022 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8463ee9-62d3-41b6-b5c9-6abe4c48a529 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:06.796308 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8463ee9-62d3-41b6-b5c9-6abe4c48a529 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:06.796574 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8463ee9-62d3-41b6-b5c9-6abe4c48a529 None admin] PATH_INFO: `/v3/users/651cdf8a382743fda784a916b79670e2` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:06.810144 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8463ee9-62d3-41b6-b5c9-6abe4c48a529 None admin] RBAC: Authorizing `identity:get_user(user_id=651cdf8a382743fda784a916b79670e2)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:18:06.811622 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8463ee9-62d3-41b6-b5c9-6abe4c48a529 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:18:06.813595 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 411/829] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Nov 2 07:18:06 2022] GET /identity/v3/users/651cdf8a382743fda784a916b79670e2 => generated 285 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:18:06.819306 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b570f19-d34a-4851-8fa3-b592d1ca262b None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:06.889273 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:18:06.922631 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b570f19-d34a-4851-8fa3-b592d1ca262b None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:06.957321 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa11577c-7fb2-45e3-ace4-5707cb54ad28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:06.957798 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa11577c-7fb2-45e3-ace4-5707cb54ad28 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:06.958078 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa11577c-7fb2-45e3-ace4-5707cb54ad28 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:06.958348 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa11577c-7fb2-45e3-ace4-5707cb54ad28 None admin] PATH_INFO: `/v3/projects/6e003375735e4a73b4afae320c3e3735` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:06.959524 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa11577c-7fb2-45e3-ace4-5707cb54ad28 None admin] RBAC: Authorizing `identity:get_project(project_id=6e003375735e4a73b4afae320c3e3735)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:18:06.960927 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa11577c-7fb2-45e3-ace4-5707cb54ad28 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:18:06.962683 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 419/830] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Nov 2 07:18:06 2022] GET /identity/v3/projects/6e003375735e4a73b4afae320c3e3735 => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:18:06.968702 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8463ee9-62d3-41b6-b5c9-6abe4c48a529 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:07.024578 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:18:07.071904 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8463ee9-62d3-41b6-b5c9-6abe4c48a529 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:07.108906 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2838a321-0d17-4e75-b183-776e3beaef46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:07.109567 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2838a321-0d17-4e75-b183-776e3beaef46 None admin] REQUEST_METHOD: `PUT` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:07.109851 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2838a321-0d17-4e75-b183-776e3beaef46 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:07.110128 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2838a321-0d17-4e75-b183-776e3beaef46 None admin] PATH_INFO: `/v3/projects/6e003375735e4a73b4afae320c3e3735/users/651cdf8a382743fda784a916b79670e2/roles/c4bf019fc6c44c71a6c984f6d0bab75f` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:07.112040 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2838a321-0d17-4e75-b183-776e3beaef46 None admin] RBAC: Authorizing `identity:create_grant(project_id=6e003375735e4a73b4afae320c3e3735, user_id=651cdf8a382743fda784a916b79670e2, role_id=c4bf019fc6c44c71a6c984f6d0bab75f)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:18:07.116993 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2838a321-0d17-4e75-b183-776e3beaef46 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:18:07.131287 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 412/831] 10.222.0.18 () {58 vars in 1439 bytes} [Wed Nov 2 07:18:06 2022] PUT /identity/v3/projects/6e003375735e4a73b4afae320c3e3735/users/651cdf8a382743fda784a916b79670e2/roles/c4bf019fc6c44c71a6c984f6d0bab75f => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:18:09.267013 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f254b44e-d801-4004-b324-866402b71c4f None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:09.267547 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f254b44e-d801-4004-b324-866402b71c4f None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:09.267547 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f254b44e-d801-4004-b324-866402b71c4f None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:09.268850 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 420/832] 10.222.0.18 () {54 vars in 927 bytes} [Wed Nov 2 07:18:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:18:09.274274 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad07808-8f75-4ea4-93cf-619e4aa0024f None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:09.274716 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad07808-8f75-4ea4-93cf-619e4aa0024f None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:09.274861 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad07808-8f75-4ea4-93cf-619e4aa0024f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:09.292553 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-dad07808-8f75-4ea4-93cf-619e4aa0024f None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:18:09.293734 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-dad07808-8f75-4ea4-93cf-619e4aa0024f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:18:09.359816 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 413/833] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Nov 2 07:18:09 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 07:18:09.628535 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f254b44e-d801-4004-b324-866402b71c4f None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:09.726063 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f254b44e-d801-4004-b324-866402b71c4f None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:09.757724 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb9ac38-39b3-49bd-9ce3-c01315450527 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:09.758330 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb9ac38-39b3-49bd-9ce3-c01315450527 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:09.758633 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb9ac38-39b3-49bd-9ce3-c01315450527 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:09.758896 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb9ac38-39b3-49bd-9ce3-c01315450527 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:09.761396 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:18:09.769681 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb9ac38-39b3-49bd-9ce3-c01315450527 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:18:09.774434 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-9bb9ac38-39b3-49bd-9ce3-c01315450527 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 07:18:09.774434 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:18:09.774434 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:18:09.774434 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:18:09.774434 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:18:09.774434 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:18:09.774434 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:18:09.774434 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:18:09.774434 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:18:09.774434 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:18:09.774434 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:18:09.774434 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:18:09.774434 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:18:09.774434 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:18:09.774434 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:18:09.774434 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:18:09.775506 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:18:09.775506 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:18:09.775506 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:18:09.775506 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:18:09.775506 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:18:09.775506 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:18:09.775506 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:18:09.775506 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:18:09.775506 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:18:09.775506 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:18:09.775506 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:18:09.775506 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:18:09.775506 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:18:09.775506 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:18:09.775506 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:18:09.775506 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:18:09.776690 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:18:09.776690 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:18:09.776690 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:18:09.776690 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:18:09.776690 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:18:09.776690 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:18:09.776690 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:18:09.776690 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:18:09.776690 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:18:09.776690 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:18:09.776690 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 07:18:09.776690 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:18:09.776690 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 421/834] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Nov 2 07:18:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:18:09.781472 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad07808-8f75-4ea4-93cf-619e4aa0024f None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:09.825893 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:18:09.878973 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad07808-8f75-4ea4-93cf-619e4aa0024f None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:09.911496 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9abd1b-b51a-4b2c-90aa-836d1d7daece None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:09.912085 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9abd1b-b51a-4b2c-90aa-836d1d7daece None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:09.912364 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9abd1b-b51a-4b2c-90aa-836d1d7daece None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:09.912634 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9abd1b-b51a-4b2c-90aa-836d1d7daece None admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:09.913281 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9abd1b-b51a-4b2c-90aa-836d1d7daece None admin] RBAC: Adding query filter params (name=admin) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:18:09.913579 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9abd1b-b51a-4b2c-90aa-836d1d7daece None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:18:09.914969 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9abd1b-b51a-4b2c-90aa-836d1d7daece None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:18:09.922664 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 414/835] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Nov 2 07:18:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:18:09.929012 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb9ac38-39b3-49bd-9ce3-c01315450527 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:10.023892 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb9ac38-39b3-49bd-9ce3-c01315450527 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:10.055821 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5460da-a295-41d6-9878-974267f47220 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:10.056417 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5460da-a295-41d6-9878-974267f47220 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:10.056689 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5460da-a295-41d6-9878-974267f47220 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:10.056953 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5460da-a295-41d6-9878-974267f47220 None admin] PATH_INFO: `/v3/users/651cdf8a382743fda784a916b79670e2` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:10.058033 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5460da-a295-41d6-9878-974267f47220 None admin] RBAC: Authorizing `identity:get_user(user_id=651cdf8a382743fda784a916b79670e2)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:18:10.059455 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5460da-a295-41d6-9878-974267f47220 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:18:10.060959 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 422/836] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Nov 2 07:18:09 2022] GET /identity/v3/users/651cdf8a382743fda784a916b79670e2 => generated 285 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:18:10.066727 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9abd1b-b51a-4b2c-90aa-836d1d7daece None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:10.161375 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9abd1b-b51a-4b2c-90aa-836d1d7daece None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:10.167775 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:18:10.193796 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e9342e-8e56-444b-a4ed-a9e1dc80348d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:10.194405 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e9342e-8e56-444b-a4ed-a9e1dc80348d None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:10.194682 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e9342e-8e56-444b-a4ed-a9e1dc80348d None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:10.194947 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e9342e-8e56-444b-a4ed-a9e1dc80348d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:10.258119 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e9342e-8e56-444b-a4ed-a9e1dc80348d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:18:10.259567 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e9342e-8e56-444b-a4ed-a9e1dc80348d None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:18:10.351520 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 415/837] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Nov 2 07:18:10 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 285 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Nov 02 07:18:10.357635 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5460da-a295-41d6-9878-974267f47220 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:10.416912 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:18:10.454745 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5460da-a295-41d6-9878-974267f47220 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:10.488356 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc26393-b96b-41d9-8b08-75eaff6ebf2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:10.488967 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc26393-b96b-41d9-8b08-75eaff6ebf2a None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:10.489238 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc26393-b96b-41d9-8b08-75eaff6ebf2a None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:10.489502 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc26393-b96b-41d9-8b08-75eaff6ebf2a None admin] PATH_INFO: `/v3/projects/6e003375735e4a73b4afae320c3e3735` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:10.490608 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc26393-b96b-41d9-8b08-75eaff6ebf2a None admin] RBAC: Authorizing `identity:get_project(project_id=6e003375735e4a73b4afae320c3e3735)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:18:10.492010 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc26393-b96b-41d9-8b08-75eaff6ebf2a None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:18:10.493576 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 423/838] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Nov 2 07:18:10 2022] GET /identity/v3/projects/6e003375735e4a73b4afae320c3e3735 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:18:10.499620 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e9342e-8e56-444b-a4ed-a9e1dc80348d None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:10.507616 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:18:10.594794 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e9342e-8e56-444b-a4ed-a9e1dc80348d None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:10.627139 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663b724f-1539-46f4-9c16-0f08ff5e3d69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:10.627724 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663b724f-1539-46f4-9c16-0f08ff5e3d69 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:10.628000 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663b724f-1539-46f4-9c16-0f08ff5e3d69 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:10.628269 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663b724f-1539-46f4-9c16-0f08ff5e3d69 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:10.629166 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663b724f-1539-46f4-9c16-0f08ff5e3d69 None admin] RBAC: Adding query filter params (role.id=c4bf019fc6c44c71a6c984f6d0bab75f, scope.project.id=6e003375735e4a73b4afae320c3e3735, user.id=651cdf8a382743fda784a916b79670e2) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:18:10.629473 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663b724f-1539-46f4-9c16-0f08ff5e3d69 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:18:10.630913 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663b724f-1539-46f4-9c16-0f08ff5e3d69 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:18:10.636408 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 416/839] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Nov 2 07:18:10 2022] GET /identity/v3/role_assignments?user.id=651cdf8a382743fda784a916b79670e2&scope.project.id=6e003375735e4a73b4afae320c3e3735&role.id=c4bf019fc6c44c71a6c984f6d0bab75f => generated 608 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:18:12.761937 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d11c96-f990-4fc6-aa8c-cd4434bf4a9d None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:12.762474 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d11c96-f990-4fc6-aa8c-cd4434bf4a9d None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:12.762474 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d11c96-f990-4fc6-aa8c-cd4434bf4a9d None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:12.763804 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 424/840] 10.222.0.18 () {54 vars in 927 bytes} [Wed Nov 2 07:18:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:18:12.768952 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33c40b9-798f-4979-b27f-20e4e5332fa3 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:12.769916 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33c40b9-798f-4979-b27f-20e4e5332fa3 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:12.770022 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33c40b9-798f-4979-b27f-20e4e5332fa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:12.788032 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-f33c40b9-798f-4979-b27f-20e4e5332fa3 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:18:12.789188 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-f33c40b9-798f-4979-b27f-20e4e5332fa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:18:12.848446 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:18:12.853579 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 417/841] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Nov 2 07:18:12 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 07:18:13.117222 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d11c96-f990-4fc6-aa8c-cd4434bf4a9d None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:13.331392 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d11c96-f990-4fc6-aa8c-cd4434bf4a9d None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:13.359277 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:18:13.364920 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdddb1e-96f5-406b-8908-821722f7ef56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:13.365529 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdddb1e-96f5-406b-8908-821722f7ef56 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:13.365800 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdddb1e-96f5-406b-8908-821722f7ef56 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:13.366062 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdddb1e-96f5-406b-8908-821722f7ef56 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:13.367284 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdddb1e-96f5-406b-8908-821722f7ef56 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:18:13.368646 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdddb1e-96f5-406b-8908-821722f7ef56 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:18:13.370162 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 425/842] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Nov 2 07:18:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:18:13.375922 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33c40b9-798f-4979-b27f-20e4e5332fa3 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:13.470418 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33c40b9-798f-4979-b27f-20e4e5332fa3 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:13.503813 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da34e150-e8e4-4458-a9e3-de93d88465da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:13.504402 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da34e150-e8e4-4458-a9e3-de93d88465da None admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:13.504680 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da34e150-e8e4-4458-a9e3-de93d88465da None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:13.504945 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da34e150-e8e4-4458-a9e3-de93d88465da None admin] PATH_INFO: `/v3/users` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:13.505582 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da34e150-e8e4-4458-a9e3-de93d88465da None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:18:13.507019 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da34e150-e8e4-4458-a9e3-de93d88465da None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:18:13.527418 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 418/843] 10.222.0.18 () {60 vars in 1132 bytes} [Wed Nov 2 07:18:13 2022] POST /identity/v3/users => generated 286 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:18:15.684499 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ca7a8c-b2f2-4a97-9f62-b15ee73ad6a0 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:15.684958 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ca7a8c-b2f2-4a97-9f62-b15ee73ad6a0 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:15.685046 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ca7a8c-b2f2-4a97-9f62-b15ee73ad6a0 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:15.686344 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 426/844] 10.222.0.18 () {54 vars in 927 bytes} [Wed Nov 2 07:18:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:18:15.692271 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5c634a-2ade-42ae-b874-0c19a9799767 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:15.692687 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5c634a-2ade-42ae-b874-0c19a9799767 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:15.692809 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5c634a-2ade-42ae-b874-0c19a9799767 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:15.710560 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-8c5c634a-2ade-42ae-b874-0c19a9799767 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:18:15.711727 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-8c5c634a-2ade-42ae-b874-0c19a9799767 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:18:15.760380 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:18:15.776958 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 419/845] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Nov 2 07:18:15 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 07:18:16.045434 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ca7a8c-b2f2-4a97-9f62-b15ee73ad6a0 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:16.143261 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ca7a8c-b2f2-4a97-9f62-b15ee73ad6a0 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:16.176042 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa6ebf6-7fcf-4ed1-b007-6cebe1e77059 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:16.176646 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa6ebf6-7fcf-4ed1-b007-6cebe1e77059 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:16.176925 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa6ebf6-7fcf-4ed1-b007-6cebe1e77059 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:16.177197 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa6ebf6-7fcf-4ed1-b007-6cebe1e77059 None admin] PATH_INFO: `/v3/roles/8cf467a4f06c44e0b77499aeababdcc3` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:16.178584 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa6ebf6-7fcf-4ed1-b007-6cebe1e77059 None admin] RBAC: Authorizing `identity:get_role(role_id=8cf467a4f06c44e0b77499aeababdcc3)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:18:16.179924 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa6ebf6-7fcf-4ed1-b007-6cebe1e77059 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:18:16.181193 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 427/846] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Nov 2 07:18:16 2022] GET /identity/v3/roles/8cf467a4f06c44e0b77499aeababdcc3 => generated 223 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:18:16.186876 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5c634a-2ade-42ae-b874-0c19a9799767 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:16.279583 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5c634a-2ade-42ae-b874-0c19a9799767 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:16.312555 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85610077-9f00-492e-8c28-ac9b87cf65ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:16.313110 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85610077-9f00-492e-8c28-ac9b87cf65ac None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:16.313388 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85610077-9f00-492e-8c28-ac9b87cf65ac None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:16.313656 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85610077-9f00-492e-8c28-ac9b87cf65ac None admin] PATH_INFO: `/v3/users/4fe0fc5b00b44431ab631722f5501fe5` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:16.326277 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85610077-9f00-492e-8c28-ac9b87cf65ac None admin] RBAC: Authorizing `identity:get_user(user_id=4fe0fc5b00b44431ab631722f5501fe5)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:18:16.327746 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85610077-9f00-492e-8c28-ac9b87cf65ac None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:18:16.329295 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 420/847] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Nov 2 07:18:16 2022] GET /identity/v3/users/4fe0fc5b00b44431ab631722f5501fe5 => generated 286 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:18:16.335083 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa6ebf6-7fcf-4ed1-b007-6cebe1e77059 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:16.399843 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:18:16.431710 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa6ebf6-7fcf-4ed1-b007-6cebe1e77059 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:16.463424 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dd18ef-eaea-4715-872f-455a72d62dd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:16.464026 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dd18ef-eaea-4715-872f-455a72d62dd2 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:16.464298 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dd18ef-eaea-4715-872f-455a72d62dd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:16.464560 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dd18ef-eaea-4715-872f-455a72d62dd2 None admin] PATH_INFO: `/v3/projects/6e003375735e4a73b4afae320c3e3735` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:16.465627 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6dd18ef-eaea-4715-872f-455a72d62dd2 None admin] RBAC: Authorizing `identity:get_project(project_id=6e003375735e4a73b4afae320c3e3735)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:18:16.467038 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6dd18ef-eaea-4715-872f-455a72d62dd2 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:18:16.468595 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 428/848] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Nov 2 07:18:16 2022] GET /identity/v3/projects/6e003375735e4a73b4afae320c3e3735 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:18:16.475181 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85610077-9f00-492e-8c28-ac9b87cf65ac None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:16.518394 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:18:16.571234 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85610077-9f00-492e-8c28-ac9b87cf65ac None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:16.604019 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e958b7-dc3d-43c6-b176-a3435000a544 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:16.604658 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e958b7-dc3d-43c6-b176-a3435000a544 None admin] REQUEST_METHOD: `PUT` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:16.604933 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e958b7-dc3d-43c6-b176-a3435000a544 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:16.605202 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e958b7-dc3d-43c6-b176-a3435000a544 None admin] PATH_INFO: `/v3/projects/6e003375735e4a73b4afae320c3e3735/users/4fe0fc5b00b44431ab631722f5501fe5/roles/8cf467a4f06c44e0b77499aeababdcc3` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:16.607280 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e958b7-dc3d-43c6-b176-a3435000a544 None admin] RBAC: Authorizing `identity:create_grant(project_id=6e003375735e4a73b4afae320c3e3735, user_id=4fe0fc5b00b44431ab631722f5501fe5, role_id=8cf467a4f06c44e0b77499aeababdcc3)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:18:16.608678 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e958b7-dc3d-43c6-b176-a3435000a544 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:18:16.620743 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 421/849] 10.222.0.18 () {58 vars in 1439 bytes} [Wed Nov 2 07:18:16 2022] PUT /identity/v3/projects/6e003375735e4a73b4afae320c3e3735/users/4fe0fc5b00b44431ab631722f5501fe5/roles/8cf467a4f06c44e0b77499aeababdcc3 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:18:18.749327 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c82797-c7ea-42f4-83f7-ed2a41fa7bdd None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:18.749878 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c82797-c7ea-42f4-83f7-ed2a41fa7bdd None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:18.749878 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c82797-c7ea-42f4-83f7-ed2a41fa7bdd None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:18.751190 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 429/850] 10.222.0.18 () {54 vars in 927 bytes} [Wed Nov 2 07:18:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:18:18.756642 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c9dc6b-2259-4709-90bd-263940e26242 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:18.757059 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c9dc6b-2259-4709-90bd-263940e26242 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:18.757182 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c9dc6b-2259-4709-90bd-263940e26242 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:18.775223 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-c5c9dc6b-2259-4709-90bd-263940e26242 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:18:18.776383 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-c5c9dc6b-2259-4709-90bd-263940e26242 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:18:18.839758 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 422/851] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Nov 2 07:18:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 07:18:19.116347 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c82797-c7ea-42f4-83f7-ed2a41fa7bdd None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:19.215286 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c82797-c7ea-42f4-83f7-ed2a41fa7bdd None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:19.248709 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc6d28a-e7dc-4f3a-a3ae-90eeb66b4ce5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:19.249317 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc6d28a-e7dc-4f3a-a3ae-90eeb66b4ce5 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:19.249592 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc6d28a-e7dc-4f3a-a3ae-90eeb66b4ce5 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:19.249859 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc6d28a-e7dc-4f3a-a3ae-90eeb66b4ce5 None admin] PATH_INFO: `/v3/roles/8cf467a4f06c44e0b77499aeababdcc3` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:19.251230 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc6d28a-e7dc-4f3a-a3ae-90eeb66b4ce5 None admin] RBAC: Authorizing `identity:get_role(role_id=8cf467a4f06c44e0b77499aeababdcc3)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:18:19.252568 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc6d28a-e7dc-4f3a-a3ae-90eeb66b4ce5 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:18:19.253919 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 430/852] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Nov 2 07:18:19 2022] GET /identity/v3/roles/8cf467a4f06c44e0b77499aeababdcc3 => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:18:19.259710 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c9dc6b-2259-4709-90bd-263940e26242 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:19.291740 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:18:19.356886 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c9dc6b-2259-4709-90bd-263940e26242 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:19.389921 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca56275-549d-42dd-8794-b731af02f8fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:19.390516 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca56275-549d-42dd-8794-b731af02f8fd None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:19.390797 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca56275-549d-42dd-8794-b731af02f8fd None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:19.391067 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca56275-549d-42dd-8794-b731af02f8fd None admin] PATH_INFO: `/v3/users/4fe0fc5b00b44431ab631722f5501fe5` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:19.392160 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca56275-549d-42dd-8794-b731af02f8fd None admin] RBAC: Authorizing `identity:get_user(user_id=4fe0fc5b00b44431ab631722f5501fe5)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:18:19.393580 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca56275-549d-42dd-8794-b731af02f8fd None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:18:19.395372 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 423/853] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Nov 2 07:18:19 2022] GET /identity/v3/users/4fe0fc5b00b44431ab631722f5501fe5 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:18:19.401079 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc6d28a-e7dc-4f3a-a3ae-90eeb66b4ce5 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:19.404768 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:18:19.497477 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc6d28a-e7dc-4f3a-a3ae-90eeb66b4ce5 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:19.529602 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdca0d3-880c-4822-9d99-d41503ecaec9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:19.530186 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdca0d3-880c-4822-9d99-d41503ecaec9 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:19.530495 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdca0d3-880c-4822-9d99-d41503ecaec9 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:19.530765 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdca0d3-880c-4822-9d99-d41503ecaec9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:19.594222 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdca0d3-880c-4822-9d99-d41503ecaec9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:18:19.595674 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdca0d3-880c-4822-9d99-d41503ecaec9 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:18:19.598289 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:18:19.692083 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 431/854] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Nov 2 07:18:19 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Nov 02 07:18:19.697905 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca56275-549d-42dd-8794-b731af02f8fd None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:19.793312 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca56275-549d-42dd-8794-b731af02f8fd None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:19.794788 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:18:19.826192 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432e4733-fc2d-4448-b731-4381b59f0a02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:19.826830 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432e4733-fc2d-4448-b731-4381b59f0a02 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:19.827104 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432e4733-fc2d-4448-b731-4381b59f0a02 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:19.827367 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432e4733-fc2d-4448-b731-4381b59f0a02 None admin] PATH_INFO: `/v3/projects/6e003375735e4a73b4afae320c3e3735` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:19.828448 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432e4733-fc2d-4448-b731-4381b59f0a02 None admin] RBAC: Authorizing `identity:get_project(project_id=6e003375735e4a73b4afae320c3e3735)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:18:19.829850 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432e4733-fc2d-4448-b731-4381b59f0a02 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:18:19.831455 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 424/855] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Nov 2 07:18:19 2022] GET /identity/v3/projects/6e003375735e4a73b4afae320c3e3735 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:18:19.838406 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdca0d3-880c-4822-9d99-d41503ecaec9 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:19.931948 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdca0d3-880c-4822-9d99-d41503ecaec9 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:19.934099 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:18:19.965032 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac29430-8034-43a9-9824-2d63b3e2e749 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:19.965609 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac29430-8034-43a9-9824-2d63b3e2e749 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:19.965880 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac29430-8034-43a9-9824-2d63b3e2e749 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:19.966140 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac29430-8034-43a9-9824-2d63b3e2e749 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:19.966869 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac29430-8034-43a9-9824-2d63b3e2e749 None admin] RBAC: Adding query filter params (role.id=8cf467a4f06c44e0b77499aeababdcc3, scope.project.id=6e003375735e4a73b4afae320c3e3735, user.id=4fe0fc5b00b44431ab631722f5501fe5) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:18:19.967164 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac29430-8034-43a9-9824-2d63b3e2e749 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:18:19.968548 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac29430-8034-43a9-9824-2d63b3e2e749 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:18:19.973976 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 432/856] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Nov 2 07:18:19 2022] GET /identity/v3/role_assignments?user.id=4fe0fc5b00b44431ab631722f5501fe5&scope.project.id=6e003375735e4a73b4afae320c3e3735&role.id=8cf467a4f06c44e0b77499aeababdcc3 => generated 608 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:18:22.106529 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c77a3ab-14d2-4a21-b94f-cf431e43c194 None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:22.107062 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c77a3ab-14d2-4a21-b94f-cf431e43c194 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:22.107062 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c77a3ab-14d2-4a21-b94f-cf431e43c194 None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:22.108393 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 425/857] 10.222.0.18 () {54 vars in 927 bytes} [Wed Nov 2 07:18:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:18:22.113532 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96eca03-6118-4712-b001-3027b8fa2bc9 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:22.113910 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96eca03-6118-4712-b001-3027b8fa2bc9 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:22.114067 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96eca03-6118-4712-b001-3027b8fa2bc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:22.132549 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-c96eca03-6118-4712-b001-3027b8fa2bc9 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:18:22.133706 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-c96eca03-6118-4712-b001-3027b8fa2bc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:18:22.197025 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 433/858] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Nov 2 07:18:22 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 07:18:22.462704 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c77a3ab-14d2-4a21-b94f-cf431e43c194 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:22.559998 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c77a3ab-14d2-4a21-b94f-cf431e43c194 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:22.592446 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e52236f-72b1-4e6b-affe-e04421f118bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:22.593064 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e52236f-72b1-4e6b-affe-e04421f118bd None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:22.593342 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e52236f-72b1-4e6b-affe-e04421f118bd None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:22.593606 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e52236f-72b1-4e6b-affe-e04421f118bd None admin] PATH_INFO: `/v3/domains/default` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:22.594713 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e52236f-72b1-4e6b-affe-e04421f118bd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:18:22.596120 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e52236f-72b1-4e6b-affe-e04421f118bd None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:18:22.597695 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 426/859] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Nov 2 07:18:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:18:22.603740 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96eca03-6118-4712-b001-3027b8fa2bc9 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:22.695577 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:18:22.699533 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96eca03-6118-4712-b001-3027b8fa2bc9 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:22.733171 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3a87ce-9459-40b6-b979-d9630e0be477 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:22.733711 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3a87ce-9459-40b6-b979-d9630e0be477 None admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:22.733986 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3a87ce-9459-40b6-b979-d9630e0be477 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:22.734252 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3a87ce-9459-40b6-b979-d9630e0be477 None admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:22.734935 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3a87ce-9459-40b6-b979-d9630e0be477 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:18:22.736345 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3a87ce-9459-40b6-b979-d9630e0be477 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:18:22.761341 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 434/860] 10.222.0.18 () {60 vars in 1141 bytes} [Wed Nov 2 07:18:22 2022] POST /identity/v3/projects => generated 311 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:18:24.930741 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d48eb4-821e-4e97-992d-037d675e6f9b None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:24.931302 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d48eb4-821e-4e97-992d-037d675e6f9b None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:24.931302 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d48eb4-821e-4e97-992d-037d675e6f9b None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:24.932541 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 427/861] 10.222.0.18 () {54 vars in 927 bytes} [Wed Nov 2 07:18:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:18:24.937813 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9665c4e-3386-4748-b3a8-971cb8999070 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:24.938224 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9665c4e-3386-4748-b3a8-971cb8999070 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:24.938352 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9665c4e-3386-4748-b3a8-971cb8999070 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:24.956548 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-a9665c4e-3386-4748-b3a8-971cb8999070 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:18:24.957964 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-a9665c4e-3386-4748-b3a8-971cb8999070 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:18:25.020490 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 435/862] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Nov 2 07:18:24 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 07:18:25.290346 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d48eb4-821e-4e97-992d-037d675e6f9b None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:25.329411 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:18:25.387422 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d48eb4-821e-4e97-992d-037d675e6f9b None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:25.419469 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f045d48-a316-4b33-850f-da7e1c19547b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:25.420084 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f045d48-a316-4b33-850f-da7e1c19547b None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:25.420358 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f045d48-a316-4b33-850f-da7e1c19547b None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:25.420624 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f045d48-a316-4b33-850f-da7e1c19547b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:25.421696 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f045d48-a316-4b33-850f-da7e1c19547b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:18:25.423127 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f045d48-a316-4b33-850f-da7e1c19547b None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:18:25.424644 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 428/863] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Nov 2 07:18:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:18:25.431001 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9665c4e-3386-4748-b3a8-971cb8999070 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:25.507647 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:18:25.528070 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9665c4e-3386-4748-b3a8-971cb8999070 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:25.561093 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc72d18-e42e-456d-8df8-899966276aad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:25.561628 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc72d18-e42e-456d-8df8-899966276aad None admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:25.561909 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc72d18-e42e-456d-8df8-899966276aad None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:25.562181 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc72d18-e42e-456d-8df8-899966276aad None admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:25.562863 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc72d18-e42e-456d-8df8-899966276aad None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:18:25.564282 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc72d18-e42e-456d-8df8-899966276aad None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:18:25.584717 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 436/864] 10.222.0.18 () {60 vars in 1132 bytes} [Wed Nov 2 07:18:25 2022] POST /identity/v3/users => generated 286 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:18:27.745648 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5001b2b1-8959-48a6-8a91-f6063bf33d3a None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:27.746167 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5001b2b1-8959-48a6-8a91-f6063bf33d3a None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:27.746251 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5001b2b1-8959-48a6-8a91-f6063bf33d3a None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:27.747587 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 429/865] 10.222.0.18 () {54 vars in 927 bytes} [Wed Nov 2 07:18:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:18:27.752638 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5bb5eb-2721-4d55-a8a1-a3ccd2e6d579 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:27.753078 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5bb5eb-2721-4d55-a8a1-a3ccd2e6d579 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:27.753171 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5bb5eb-2721-4d55-a8a1-a3ccd2e6d579 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:27.771931 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-8e5bb5eb-2721-4d55-a8a1-a3ccd2e6d579 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:18:27.773076 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-8e5bb5eb-2721-4d55-a8a1-a3ccd2e6d579 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:18:27.835338 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 437/866] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Nov 2 07:18:27 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 07:18:28.105446 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5001b2b1-8959-48a6-8a91-f6063bf33d3a None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:28.202313 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5001b2b1-8959-48a6-8a91-f6063bf33d3a None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:28.209826 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:18:28.235451 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3043d51-897d-446a-940a-9c4cccdfd2c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:28.236058 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3043d51-897d-446a-940a-9c4cccdfd2c0 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:28.236333 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3043d51-897d-446a-940a-9c4cccdfd2c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:28.236595 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3043d51-897d-446a-940a-9c4cccdfd2c0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:28.246135 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3043d51-897d-446a-940a-9c4cccdfd2c0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:18:28.250901 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-e3043d51-897d-446a-940a-9c4cccdfd2c0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 07:18:28.250901 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:18:28.250901 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:18:28.250901 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:18:28.250901 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:18:28.250901 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:18:28.250901 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:18:28.250901 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:18:28.250901 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:18:28.250901 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:18:28.250901 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:18:28.250901 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:18:28.250901 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:18:28.250901 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:18:28.250901 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:18:28.250901 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:18:28.251971 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:18:28.251971 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:18:28.251971 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:18:28.251971 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:18:28.251971 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:18:28.251971 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:18:28.251971 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:18:28.251971 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:18:28.251971 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:18:28.251971 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:18:28.251971 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:18:28.251971 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Nov 02 07:18:28.251971 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:18:28.251971 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:18:28.251971 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:18:28.251971 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:18:28.252999 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:18:28.252999 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:18:28.252999 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:18:28.252999 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:18:28.252999 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:18:28.252999 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:18:28.252999 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:18:28.252999 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:18:28.252999 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:18:28.252999 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:18:28.252999 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 07:18:28.252999 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Nov 02 07:18:28.252999 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 430/867] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Nov 2 07:18:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:18:28.258554 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5bb5eb-2721-4d55-a8a1-a3ccd2e6d579 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:28.320017 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:18:28.355795 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5bb5eb-2721-4d55-a8a1-a3ccd2e6d579 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:28.387686 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3557bdd1-85c1-49ff-8ce1-8daab2a71594 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:28.388270 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3557bdd1-85c1-49ff-8ce1-8daab2a71594 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:28.388542 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3557bdd1-85c1-49ff-8ce1-8daab2a71594 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:28.388805 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3557bdd1-85c1-49ff-8ce1-8daab2a71594 None admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:28.389443 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3557bdd1-85c1-49ff-8ce1-8daab2a71594 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:18:28.389735 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3557bdd1-85c1-49ff-8ce1-8daab2a71594 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:18:28.391094 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3557bdd1-85c1-49ff-8ce1-8daab2a71594 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:18:28.399170 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 438/868] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Nov 2 07:18:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:18:28.405434 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3043d51-897d-446a-940a-9c4cccdfd2c0 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:28.499875 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3043d51-897d-446a-940a-9c4cccdfd2c0 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:28.531671 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c0b154-0cd5-4440-8cdf-bc9f696bb470 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:28.532271 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c0b154-0cd5-4440-8cdf-bc9f696bb470 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:28.532546 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c0b154-0cd5-4440-8cdf-bc9f696bb470 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:28.532811 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c0b154-0cd5-4440-8cdf-bc9f696bb470 None admin] PATH_INFO: `/v3/users/eb43bf52e6ed434eb88a96507bbeac1d` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:28.545130 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c0b154-0cd5-4440-8cdf-bc9f696bb470 None admin] RBAC: Authorizing `identity:get_user(user_id=eb43bf52e6ed434eb88a96507bbeac1d)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:18:28.546570 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c0b154-0cd5-4440-8cdf-bc9f696bb470 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:18:28.548122 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 431/869] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Nov 2 07:18:28 2022] GET /identity/v3/users/eb43bf52e6ed434eb88a96507bbeac1d => generated 286 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:18:28.554587 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3557bdd1-85c1-49ff-8ce1-8daab2a71594 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:28.651958 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3557bdd1-85c1-49ff-8ce1-8daab2a71594 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:28.677182 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:18:28.685227 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98ee099-e319-41f2-9cc1-6b940c1fcc56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:28.685843 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98ee099-e319-41f2-9cc1-6b940c1fcc56 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:28.686119 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98ee099-e319-41f2-9cc1-6b940c1fcc56 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:28.686407 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98ee099-e319-41f2-9cc1-6b940c1fcc56 None admin] PATH_INFO: `/v3/projects/bb6d99d2596f4fb3a8ed0e459f3d0cec` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:28.687498 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98ee099-e319-41f2-9cc1-6b940c1fcc56 None admin] RBAC: Authorizing `identity:get_project(project_id=bb6d99d2596f4fb3a8ed0e459f3d0cec)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:18:28.688899 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98ee099-e319-41f2-9cc1-6b940c1fcc56 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:18:28.690502 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 439/870] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Nov 2 07:18:28 2022] GET /identity/v3/projects/bb6d99d2596f4fb3a8ed0e459f3d0cec => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:18:28.697009 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c0b154-0cd5-4440-8cdf-bc9f696bb470 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:28.721578 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:18:28.794503 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c0b154-0cd5-4440-8cdf-bc9f696bb470 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:28.827076 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a146b2-3dcb-4d9b-9060-660a610b9290 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:28.827721 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a146b2-3dcb-4d9b-9060-660a610b9290 None admin] REQUEST_METHOD: `PUT` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:28.828000 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a146b2-3dcb-4d9b-9060-660a610b9290 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:28.828269 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a146b2-3dcb-4d9b-9060-660a610b9290 None admin] PATH_INFO: `/v3/projects/bb6d99d2596f4fb3a8ed0e459f3d0cec/users/eb43bf52e6ed434eb88a96507bbeac1d/roles/c4bf019fc6c44c71a6c984f6d0bab75f` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:28.830129 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a146b2-3dcb-4d9b-9060-660a610b9290 None admin] RBAC: Authorizing `identity:create_grant(project_id=bb6d99d2596f4fb3a8ed0e459f3d0cec, user_id=eb43bf52e6ed434eb88a96507bbeac1d, role_id=c4bf019fc6c44c71a6c984f6d0bab75f)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:18:28.831551 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a146b2-3dcb-4d9b-9060-660a610b9290 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:18:28.844074 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 432/871] 10.222.0.18 () {58 vars in 1439 bytes} [Wed Nov 2 07:18:28 2022] PUT /identity/v3/projects/bb6d99d2596f4fb3a8ed0e459f3d0cec/users/eb43bf52e6ed434eb88a96507bbeac1d/roles/c4bf019fc6c44c71a6c984f6d0bab75f => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:18:30.941609 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3bb64e-3d33-428c-9c3b-e391dbc1cf12 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:30.942173 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3bb64e-3d33-428c-9c3b-e391dbc1cf12 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:30.942274 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3bb64e-3d33-428c-9c3b-e391dbc1cf12 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:30.943641 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 440/872] 10.222.0.18 () {54 vars in 927 bytes} [Wed Nov 2 07:18:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:18:30.948799 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62efef0-e901-48f8-bd6c-adc5a1df2377 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:30.949207 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62efef0-e901-48f8-bd6c-adc5a1df2377 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:30.949354 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62efef0-e901-48f8-bd6c-adc5a1df2377 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:30.968090 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-b62efef0-e901-48f8-bd6c-adc5a1df2377 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:18:30.969275 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-b62efef0-e901-48f8-bd6c-adc5a1df2377 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:18:31.034888 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 433/873] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Nov 2 07:18:30 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Nov 02 07:18:31.305146 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3bb64e-3d33-428c-9c3b-e391dbc1cf12 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:31.405769 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3bb64e-3d33-428c-9c3b-e391dbc1cf12 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:31.438297 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be725b3f-344d-46c8-b9a4-cf53513f1154 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:31.438940 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be725b3f-344d-46c8-b9a4-cf53513f1154 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:31.439219 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be725b3f-344d-46c8-b9a4-cf53513f1154 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:31.439491 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be725b3f-344d-46c8-b9a4-cf53513f1154 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:31.449670 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be725b3f-344d-46c8-b9a4-cf53513f1154 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:18:31.454446 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-be725b3f-344d-46c8-b9a4-cf53513f1154 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 07:18:31.454446 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:18:31.454446 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:18:31.454446 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:18:31.454446 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:18:31.454446 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:18:31.454446 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:18:31.454446 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:18:31.454446 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:18:31.454446 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:18:31.454446 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:18:31.454446 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:18:31.454446 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:18:31.454446 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:18:31.454446 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:18:31.454446 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:18:31.455515 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:18:31.455515 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:18:31.455515 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:18:31.455515 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:18:31.455515 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:18:31.455515 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:18:31.455515 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:18:31.455515 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:18:31.455515 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:18:31.455515 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:18:31.455515 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:18:31.455515 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:18:31.455515 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:18:31.455515 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:18:31.455515 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:18:31.455515 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:18:31.456372 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:18:31.456372 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:18:31.456372 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:18:31.456372 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:18:31.456372 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:18:31.456372 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:18:31.456372 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:18:31.456372 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:18:31.456372 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:18:31.456372 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:18:31.456372 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 07:18:31.456372 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:18:31.456372 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 441/874] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Nov 2 07:18:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:18:31.462145 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62efef0-e901-48f8-bd6c-adc5a1df2377 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:31.477107 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:18:31.557867 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62efef0-e901-48f8-bd6c-adc5a1df2377 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:31.589722 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b48a60-c404-4eab-a93a-56c3a2917663 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:31.590301 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b48a60-c404-4eab-a93a-56c3a2917663 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:31.590603 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b48a60-c404-4eab-a93a-56c3a2917663 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:31.590868 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b48a60-c404-4eab-a93a-56c3a2917663 None admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:31.591512 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b48a60-c404-4eab-a93a-56c3a2917663 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:18:31.591805 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b48a60-c404-4eab-a93a-56c3a2917663 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:18:31.593145 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b48a60-c404-4eab-a93a-56c3a2917663 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:18:31.600886 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 434/875] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Nov 2 07:18:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:18:31.607065 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be725b3f-344d-46c8-b9a4-cf53513f1154 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:31.660776 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:18:31.703157 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be725b3f-344d-46c8-b9a4-cf53513f1154 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:31.735830 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabea46a-1249-4614-8944-087f175d5b40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:31.736435 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabea46a-1249-4614-8944-087f175d5b40 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:31.736712 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabea46a-1249-4614-8944-087f175d5b40 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:31.736979 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabea46a-1249-4614-8944-087f175d5b40 None admin] PATH_INFO: `/v3/users/eb43bf52e6ed434eb88a96507bbeac1d` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:31.738065 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabea46a-1249-4614-8944-087f175d5b40 None admin] RBAC: Authorizing `identity:get_user(user_id=eb43bf52e6ed434eb88a96507bbeac1d)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:18:31.739650 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabea46a-1249-4614-8944-087f175d5b40 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:18:31.741178 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 442/876] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Nov 2 07:18:31 2022] GET /identity/v3/users/eb43bf52e6ed434eb88a96507bbeac1d => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:18:31.746929 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b48a60-c404-4eab-a93a-56c3a2917663 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:31.821528 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:18:31.844050 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b48a60-c404-4eab-a93a-56c3a2917663 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:31.875638 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8f5544-f3a8-4c2d-a0b7-b70cc812b0f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:31.876230 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8f5544-f3a8-4c2d-a0b7-b70cc812b0f7 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:31.876508 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8f5544-f3a8-4c2d-a0b7-b70cc812b0f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:31.876778 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8f5544-f3a8-4c2d-a0b7-b70cc812b0f7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:31.940296 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8f5544-f3a8-4c2d-a0b7-b70cc812b0f7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:18:31.941727 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8f5544-f3a8-4c2d-a0b7-b70cc812b0f7 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:18:32.013199 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:18:32.036700 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 435/877] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Nov 2 07:18:31 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Nov 02 07:18:32.042662 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabea46a-1249-4614-8944-087f175d5b40 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:32.139339 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabea46a-1249-4614-8944-087f175d5b40 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:32.162618 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:18:32.173130 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06558530-176d-4d48-a2ad-0d3620dc9171 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:32.173746 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06558530-176d-4d48-a2ad-0d3620dc9171 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:32.174020 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06558530-176d-4d48-a2ad-0d3620dc9171 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:32.174284 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06558530-176d-4d48-a2ad-0d3620dc9171 None admin] PATH_INFO: `/v3/projects/bb6d99d2596f4fb3a8ed0e459f3d0cec` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:32.175401 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06558530-176d-4d48-a2ad-0d3620dc9171 None admin] RBAC: Authorizing `identity:get_project(project_id=bb6d99d2596f4fb3a8ed0e459f3d0cec)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:18:32.177015 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06558530-176d-4d48-a2ad-0d3620dc9171 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:18:32.178637 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 443/878] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Nov 2 07:18:32 2022] GET /identity/v3/projects/bb6d99d2596f4fb3a8ed0e459f3d0cec => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:18:32.184689 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8f5544-f3a8-4c2d-a0b7-b70cc812b0f7 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:32.279546 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8f5544-f3a8-4c2d-a0b7-b70cc812b0f7 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:32.312665 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977d7b06-351c-4e43-952b-78848706db6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:32.313249 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977d7b06-351c-4e43-952b-78848706db6c None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:32.313522 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977d7b06-351c-4e43-952b-78848706db6c None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:32.313785 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977d7b06-351c-4e43-952b-78848706db6c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:32.314515 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977d7b06-351c-4e43-952b-78848706db6c None admin] RBAC: Adding query filter params (role.id=c4bf019fc6c44c71a6c984f6d0bab75f, scope.project.id=bb6d99d2596f4fb3a8ed0e459f3d0cec, user.id=eb43bf52e6ed434eb88a96507bbeac1d) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:18:32.314819 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977d7b06-351c-4e43-952b-78848706db6c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:18:32.316213 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977d7b06-351c-4e43-952b-78848706db6c None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:18:32.322042 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 436/879] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Nov 2 07:18:32 2022] GET /identity/v3/role_assignments?user.id=eb43bf52e6ed434eb88a96507bbeac1d&scope.project.id=bb6d99d2596f4fb3a8ed0e459f3d0cec&role.id=c4bf019fc6c44c71a6c984f6d0bab75f => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:18:34.447779 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cd0a09-0194-4511-8ce0-1e40bb15835d None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:34.448367 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cd0a09-0194-4511-8ce0-1e40bb15835d None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:34.448367 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cd0a09-0194-4511-8ce0-1e40bb15835d None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:34.449615 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 444/880] 10.222.0.18 () {54 vars in 927 bytes} [Wed Nov 2 07:18:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:18:34.455079 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca90eb8e-0bb1-4a89-8fc7-ae036d78a45e None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:34.455511 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca90eb8e-0bb1-4a89-8fc7-ae036d78a45e None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:34.455617 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca90eb8e-0bb1-4a89-8fc7-ae036d78a45e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:34.474233 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-ca90eb8e-0bb1-4a89-8fc7-ae036d78a45e None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:18:34.475423 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-ca90eb8e-0bb1-4a89-8fc7-ae036d78a45e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:18:34.507868 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:18:34.539982 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 437/881] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Nov 2 07:18:34 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Nov 02 07:18:34.812848 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cd0a09-0194-4511-8ce0-1e40bb15835d None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:34.910347 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cd0a09-0194-4511-8ce0-1e40bb15835d None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:34.941046 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12afad22-283b-4c43-852e-f73a2606c3ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:34.941627 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12afad22-283b-4c43-852e-f73a2606c3ff None admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:34.942444 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12afad22-283b-4c43-852e-f73a2606c3ff None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:34.942701 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12afad22-283b-4c43-852e-f73a2606c3ff None admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:34.943347 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12afad22-283b-4c43-852e-f73a2606c3ff None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:18:34.944715 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12afad22-283b-4c43-852e-f73a2606c3ff None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:18:34.960302 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 445/882] 10.222.0.18 () {60 vars in 1138 bytes} [Wed Nov 2 07:18:34 2022] POST /identity/v3/domains => generated 267 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:18:37.122461 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af6ead7-1066-407c-99fd-e85c231d2c25 None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:37.123017 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af6ead7-1066-407c-99fd-e85c231d2c25 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:37.123017 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af6ead7-1066-407c-99fd-e85c231d2c25 None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:37.124339 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 438/883] 10.222.0.18 () {54 vars in 927 bytes} [Wed Nov 2 07:18:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:18:37.130151 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e8b582-323f-4243-8796-5737824f1ee5 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:37.130662 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e8b582-323f-4243-8796-5737824f1ee5 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:37.130758 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e8b582-323f-4243-8796-5737824f1ee5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:37.149008 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-55e8b582-323f-4243-8796-5737824f1ee5 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:18:37.150179 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-55e8b582-323f-4243-8796-5737824f1ee5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:18:37.195277 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:18:37.214806 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 446/884] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Nov 2 07:18:37 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 07:18:37.491731 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af6ead7-1066-407c-99fd-e85c231d2c25 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:37.588726 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af6ead7-1066-407c-99fd-e85c231d2c25 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:37.621020 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b787be61-7c77-4108-a209-1c991c8ccdbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:37.621631 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b787be61-7c77-4108-a209-1c991c8ccdbc None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:37.621908 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b787be61-7c77-4108-a209-1c991c8ccdbc None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:37.622179 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b787be61-7c77-4108-a209-1c991c8ccdbc None admin] PATH_INFO: `/v3/domains/53015a467c6b4d9b9b02de9e237dc02d` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:37.632145 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b787be61-7c77-4108-a209-1c991c8ccdbc None admin] RBAC: Authorizing `identity:get_domain(domain_id=53015a467c6b4d9b9b02de9e237dc02d)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:18:37.633565 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b787be61-7c77-4108-a209-1c991c8ccdbc None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:18:37.635132 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 439/885] 10.222.0.18 () {56 vars in 1183 bytes} [Wed Nov 2 07:18:37 2022] GET /identity/v3/domains/53015a467c6b4d9b9b02de9e237dc02d => generated 267 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:18:37.641039 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e8b582-323f-4243-8796-5737824f1ee5 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:37.735128 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e8b582-323f-4243-8796-5737824f1ee5 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:37.766319 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f7a74a-64ed-45db-b321-fb0795ec8762 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:37.766937 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f7a74a-64ed-45db-b321-fb0795ec8762 None admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:37.767211 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f7a74a-64ed-45db-b321-fb0795ec8762 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:37.767475 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f7a74a-64ed-45db-b321-fb0795ec8762 None admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:37.768120 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f7a74a-64ed-45db-b321-fb0795ec8762 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:18:37.769510 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f7a74a-64ed-45db-b321-fb0795ec8762 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:18:37.797241 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 447/886] 10.222.0.18 () {60 vars in 1141 bytes} [Wed Nov 2 07:18:37 2022] POST /identity/v3/projects => generated 361 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:18:39.949368 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8e303e-37bf-415f-accd-842cca3f03ff None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:39.949887 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8e303e-37bf-415f-accd-842cca3f03ff None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:39.949887 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8e303e-37bf-415f-accd-842cca3f03ff None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:39.951214 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 440/887] 10.222.0.18 () {54 vars in 927 bytes} [Wed Nov 2 07:18:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:18:39.956485 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de48e40-43a5-4a1d-8059-c02594b2f0ec None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:39.956907 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de48e40-43a5-4a1d-8059-c02594b2f0ec None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:39.957007 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de48e40-43a5-4a1d-8059-c02594b2f0ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:39.974724 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-4de48e40-43a5-4a1d-8059-c02594b2f0ec None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:18:39.975873 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-4de48e40-43a5-4a1d-8059-c02594b2f0ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:18:40.001412 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:18:40.041681 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 448/888] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Nov 2 07:18:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 07:18:40.311215 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8e303e-37bf-415f-accd-842cca3f03ff None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:40.343551 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:18:40.415484 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8e303e-37bf-415f-accd-842cca3f03ff None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:40.447823 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976f8b86-97db-4358-a71d-88e70e43b5fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:40.448438 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976f8b86-97db-4358-a71d-88e70e43b5fc None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:40.448714 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976f8b86-97db-4358-a71d-88e70e43b5fc None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:40.448983 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976f8b86-97db-4358-a71d-88e70e43b5fc None admin] PATH_INFO: `/v3/domains/53015a467c6b4d9b9b02de9e237dc02d` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:40.450066 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976f8b86-97db-4358-a71d-88e70e43b5fc None admin] RBAC: Authorizing `identity:get_domain(domain_id=53015a467c6b4d9b9b02de9e237dc02d)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:18:40.451507 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976f8b86-97db-4358-a71d-88e70e43b5fc None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:18:40.453051 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 441/889] 10.222.0.18 () {56 vars in 1183 bytes} [Wed Nov 2 07:18:40 2022] GET /identity/v3/domains/53015a467c6b4d9b9b02de9e237dc02d => generated 267 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:18:40.458844 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de48e40-43a5-4a1d-8059-c02594b2f0ec None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:40.552466 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de48e40-43a5-4a1d-8059-c02594b2f0ec None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:40.584679 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa1ec95-63d9-48dc-a38d-e5b6d0273cf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:40.585207 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa1ec95-63d9-48dc-a38d-e5b6d0273cf2 None admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:40.585483 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa1ec95-63d9-48dc-a38d-e5b6d0273cf2 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:40.585750 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa1ec95-63d9-48dc-a38d-e5b6d0273cf2 None admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:40.586413 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa1ec95-63d9-48dc-a38d-e5b6d0273cf2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:18:40.587820 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa1ec95-63d9-48dc-a38d-e5b6d0273cf2 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:18:40.608746 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 449/890] 10.222.0.18 () {60 vars in 1132 bytes} [Wed Nov 2 07:18:40 2022] POST /identity/v3/users => generated 311 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:18:42.900455 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1ef01d-87e3-4c5a-b03c-6243f958b304 None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:42.901009 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1ef01d-87e3-4c5a-b03c-6243f958b304 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:42.901009 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1ef01d-87e3-4c5a-b03c-6243f958b304 None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:42.902270 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 442/891] 10.222.0.18 () {54 vars in 927 bytes} [Wed Nov 2 07:18:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:18:42.907518 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14aa0b97-0ee1-4bb0-b5cb-2ba3b48ceaa5 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:42.907945 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14aa0b97-0ee1-4bb0-b5cb-2ba3b48ceaa5 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:42.908063 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14aa0b97-0ee1-4bb0-b5cb-2ba3b48ceaa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:42.926435 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-14aa0b97-0ee1-4bb0-b5cb-2ba3b48ceaa5 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:18:42.927602 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-14aa0b97-0ee1-4bb0-b5cb-2ba3b48ceaa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:18:42.940777 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:18:42.994039 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 450/892] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Nov 2 07:18:42 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 07:18:43.261043 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1ef01d-87e3-4c5a-b03c-6243f958b304 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:43.356452 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:18:43.361358 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1ef01d-87e3-4c5a-b03c-6243f958b304 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:43.393252 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4310ed-ae2d-40e8-a4ce-297c8891723d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:43.393864 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4310ed-ae2d-40e8-a4ce-297c8891723d None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:43.394137 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4310ed-ae2d-40e8-a4ce-297c8891723d None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:43.394426 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4310ed-ae2d-40e8-a4ce-297c8891723d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:43.405203 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4310ed-ae2d-40e8-a4ce-297c8891723d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:18:43.409844 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-fd4310ed-ae2d-40e8-a4ce-297c8891723d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 07:18:43.409844 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:18:43.409844 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:18:43.409844 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:18:43.409844 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:18:43.409844 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:18:43.409844 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:18:43.409844 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:18:43.409844 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:18:43.409844 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:18:43.409844 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:18:43.409844 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:18:43.409844 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:18:43.409844 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:18:43.409844 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:18:43.409844 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:18:43.410982 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:18:43.410982 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:18:43.410982 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:18:43.410982 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:18:43.410982 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:18:43.410982 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:18:43.410982 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:18:43.410982 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:18:43.410982 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:18:43.410982 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:18:43.410982 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:18:43.410982 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Nov 02 07:18:43.410982 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:18:43.410982 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:18:43.410982 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:18:43.410982 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:18:43.412087 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:18:43.412087 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:18:43.412087 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:18:43.412087 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:18:43.412087 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:18:43.412087 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:18:43.412087 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:18:43.412087 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:18:43.412087 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:18:43.412087 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:18:43.412087 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 07:18:43.412087 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Nov 02 07:18:43.412087 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 443/893] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Nov 2 07:18:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:18:43.417124 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14aa0b97-0ee1-4bb0-b5cb-2ba3b48ceaa5 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:43.510991 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14aa0b97-0ee1-4bb0-b5cb-2ba3b48ceaa5 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:43.543245 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55123edc-7f74-4952-8ab3-aa924d7cb080 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:43.543773 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55123edc-7f74-4952-8ab3-aa924d7cb080 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:43.544046 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55123edc-7f74-4952-8ab3-aa924d7cb080 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:43.544312 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55123edc-7f74-4952-8ab3-aa924d7cb080 None admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:43.544954 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55123edc-7f74-4952-8ab3-aa924d7cb080 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:18:43.545246 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55123edc-7f74-4952-8ab3-aa924d7cb080 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:18:43.546611 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55123edc-7f74-4952-8ab3-aa924d7cb080 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:18:43.554276 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 451/894] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Nov 2 07:18:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:18:43.559994 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4310ed-ae2d-40e8-a4ce-297c8891723d None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:43.657357 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4310ed-ae2d-40e8-a4ce-297c8891723d None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:43.689270 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1e7404-1cdd-49ef-af5d-30115bf89b6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:43.689875 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1e7404-1cdd-49ef-af5d-30115bf89b6f None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:43.690153 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1e7404-1cdd-49ef-af5d-30115bf89b6f None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:43.690450 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1e7404-1cdd-49ef-af5d-30115bf89b6f None admin] PATH_INFO: `/v3/users/2cdd0cb043f94082854b2a11105255e6` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:43.704329 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1e7404-1cdd-49ef-af5d-30115bf89b6f None admin] RBAC: Authorizing `identity:get_user(user_id=2cdd0cb043f94082854b2a11105255e6)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:18:43.705740 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1e7404-1cdd-49ef-af5d-30115bf89b6f None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:18:43.707333 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 444/895] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Nov 2 07:18:43 2022] GET /identity/v3/users/2cdd0cb043f94082854b2a11105255e6 => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:18:43.713058 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55123edc-7f74-4952-8ab3-aa924d7cb080 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:43.727891 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:18:43.809464 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55123edc-7f74-4952-8ab3-aa924d7cb080 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:43.840766 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53ca6f0-704b-40f5-aae4-5ecb34711de8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:43.841377 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53ca6f0-704b-40f5-aae4-5ecb34711de8 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:43.841706 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53ca6f0-704b-40f5-aae4-5ecb34711de8 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:43.841979 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53ca6f0-704b-40f5-aae4-5ecb34711de8 None admin] PATH_INFO: `/v3/projects/7a48fb6b31f94e42b655fb380713f180` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:43.843095 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53ca6f0-704b-40f5-aae4-5ecb34711de8 None admin] RBAC: Authorizing `identity:get_project(project_id=7a48fb6b31f94e42b655fb380713f180)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:18:43.844498 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53ca6f0-704b-40f5-aae4-5ecb34711de8 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:18:43.846063 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 452/896] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Nov 2 07:18:43 2022] GET /identity/v3/projects/7a48fb6b31f94e42b655fb380713f180 => generated 361 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:18:43.852427 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1e7404-1cdd-49ef-af5d-30115bf89b6f None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:43.870691 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:18:43.948862 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1e7404-1cdd-49ef-af5d-30115bf89b6f None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:43.981933 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3d986d-fd57-47bd-85b9-a37d673d3bae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:43.982597 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3d986d-fd57-47bd-85b9-a37d673d3bae None admin] REQUEST_METHOD: `PUT` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:43.982874 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3d986d-fd57-47bd-85b9-a37d673d3bae None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:43.983142 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3d986d-fd57-47bd-85b9-a37d673d3bae None admin] PATH_INFO: `/v3/projects/7a48fb6b31f94e42b655fb380713f180/users/2cdd0cb043f94082854b2a11105255e6/roles/c4bf019fc6c44c71a6c984f6d0bab75f` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:43.984984 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3d986d-fd57-47bd-85b9-a37d673d3bae None admin] RBAC: Authorizing `identity:create_grant(project_id=7a48fb6b31f94e42b655fb380713f180, user_id=2cdd0cb043f94082854b2a11105255e6, role_id=c4bf019fc6c44c71a6c984f6d0bab75f)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:18:43.986410 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3d986d-fd57-47bd-85b9-a37d673d3bae None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:18:43.998652 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 445/897] 10.222.0.18 () {58 vars in 1439 bytes} [Wed Nov 2 07:18:43 2022] PUT /identity/v3/projects/7a48fb6b31f94e42b655fb380713f180/users/2cdd0cb043f94082854b2a11105255e6/roles/c4bf019fc6c44c71a6c984f6d0bab75f => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:18:46.103838 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69c3adc-551e-48c5-9b44-ab870a94007d None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:46.104374 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69c3adc-551e-48c5-9b44-ab870a94007d None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:46.104374 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69c3adc-551e-48c5-9b44-ab870a94007d None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:46.105708 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 453/898] 10.222.0.18 () {54 vars in 927 bytes} [Wed Nov 2 07:18:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:18:46.111058 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34483d7-fc4b-4413-b7eb-4cf23e91beca None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:46.111478 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34483d7-fc4b-4413-b7eb-4cf23e91beca None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:46.111606 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34483d7-fc4b-4413-b7eb-4cf23e91beca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:46.130092 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-f34483d7-fc4b-4413-b7eb-4cf23e91beca None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:18:46.131297 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-f34483d7-fc4b-4413-b7eb-4cf23e91beca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:18:46.195466 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 446/899] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Nov 2 07:18:46 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Nov 02 07:18:46.463119 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69c3adc-551e-48c5-9b44-ab870a94007d None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:46.541453 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:18:46.561173 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69c3adc-551e-48c5-9b44-ab870a94007d None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:46.592991 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b937fe1-c87c-4b7c-bd19-9641b09a06ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:46.593600 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b937fe1-c87c-4b7c-bd19-9641b09a06ad None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:46.593877 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b937fe1-c87c-4b7c-bd19-9641b09a06ad None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:46.594145 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b937fe1-c87c-4b7c-bd19-9641b09a06ad None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:46.604521 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b937fe1-c87c-4b7c-bd19-9641b09a06ad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:18:46.609246 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-6b937fe1-c87c-4b7c-bd19-9641b09a06ad None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 07:18:46.609246 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:18:46.609246 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:18:46.609246 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:18:46.609246 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:18:46.609246 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:18:46.609246 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:18:46.609246 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:18:46.609246 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:18:46.609246 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:18:46.609246 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:18:46.609246 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:18:46.609246 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:18:46.609246 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:18:46.609246 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:18:46.609246 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:18:46.610314 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:18:46.610314 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:18:46.610314 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:18:46.610314 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:18:46.610314 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:18:46.610314 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:18:46.610314 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:18:46.610314 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:18:46.610314 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:18:46.610314 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:18:46.610314 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:18:46.610314 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:18:46.610314 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:18:46.610314 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:18:46.610314 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:18:46.610314 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:18:46.611302 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:18:46.611302 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:18:46.611302 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:18:46.611302 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:18:46.611302 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:18:46.611302 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:18:46.611302 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:18:46.611302 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:18:46.611302 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:18:46.611302 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:18:46.611302 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 07:18:46.611302 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:18:46.611302 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 454/900] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Nov 2 07:18:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:18:46.617101 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34483d7-fc4b-4413-b7eb-4cf23e91beca None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:46.625076 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:18:46.716739 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34483d7-fc4b-4413-b7eb-4cf23e91beca None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:46.749778 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18fadd3-2ca4-4eac-ae0b-d6775b39fda7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:46.750396 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18fadd3-2ca4-4eac-ae0b-d6775b39fda7 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:46.750682 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18fadd3-2ca4-4eac-ae0b-d6775b39fda7 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:46.750953 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18fadd3-2ca4-4eac-ae0b-d6775b39fda7 None admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:46.751606 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18fadd3-2ca4-4eac-ae0b-d6775b39fda7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:18:46.751907 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18fadd3-2ca4-4eac-ae0b-d6775b39fda7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:18:46.753266 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18fadd3-2ca4-4eac-ae0b-d6775b39fda7 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:18:46.762006 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 447/901] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Nov 2 07:18:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:18:46.767982 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b937fe1-c87c-4b7c-bd19-9641b09a06ad None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:46.864579 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b937fe1-c87c-4b7c-bd19-9641b09a06ad None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:46.895996 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1807ad14-da23-4bd1-90b9-e35662c2e82d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:46.896598 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1807ad14-da23-4bd1-90b9-e35662c2e82d None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:46.896874 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1807ad14-da23-4bd1-90b9-e35662c2e82d None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:46.897140 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1807ad14-da23-4bd1-90b9-e35662c2e82d None admin] PATH_INFO: `/v3/users/2cdd0cb043f94082854b2a11105255e6` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:46.898216 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1807ad14-da23-4bd1-90b9-e35662c2e82d None admin] RBAC: Authorizing `identity:get_user(user_id=2cdd0cb043f94082854b2a11105255e6)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:18:46.899640 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1807ad14-da23-4bd1-90b9-e35662c2e82d None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:18:46.901147 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 455/902] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Nov 2 07:18:46 2022] GET /identity/v3/users/2cdd0cb043f94082854b2a11105255e6 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:18:46.906839 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18fadd3-2ca4-4eac-ae0b-d6775b39fda7 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:46.970411 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:18:47.003230 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18fadd3-2ca4-4eac-ae0b-d6775b39fda7 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:47.035181 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f81d31-1af2-4942-a7fb-29fa292726e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:47.036350 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f81d31-1af2-4942-a7fb-29fa292726e7 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:47.036630 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f81d31-1af2-4942-a7fb-29fa292726e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:47.036891 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f81d31-1af2-4942-a7fb-29fa292726e7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:47.100869 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f81d31-1af2-4942-a7fb-29fa292726e7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:18:47.102283 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f81d31-1af2-4942-a7fb-29fa292726e7 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:18:47.164174 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:18:47.197200 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 448/903] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Nov 2 07:18:46 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Nov 02 07:18:47.203180 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1807ad14-da23-4bd1-90b9-e35662c2e82d None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:47.214734 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:18:47.303023 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1807ad14-da23-4bd1-90b9-e35662c2e82d None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:47.334814 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a5a73c-e63d-4367-805d-a2edbfebe127 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:47.335422 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a5a73c-e63d-4367-805d-a2edbfebe127 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:47.335696 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a5a73c-e63d-4367-805d-a2edbfebe127 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:47.335964 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a5a73c-e63d-4367-805d-a2edbfebe127 None admin] PATH_INFO: `/v3/projects/7a48fb6b31f94e42b655fb380713f180` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:47.337037 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a5a73c-e63d-4367-805d-a2edbfebe127 None admin] RBAC: Authorizing `identity:get_project(project_id=7a48fb6b31f94e42b655fb380713f180)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:18:47.338463 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a5a73c-e63d-4367-805d-a2edbfebe127 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:18:47.340035 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 456/904] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Nov 2 07:18:47 2022] GET /identity/v3/projects/7a48fb6b31f94e42b655fb380713f180 => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:18:47.346222 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f81d31-1af2-4942-a7fb-29fa292726e7 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:47.440925 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f81d31-1af2-4942-a7fb-29fa292726e7 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:47.472887 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7b64f3-1142-4db3-9be9-5a786809d814 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:47.473566 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7b64f3-1142-4db3-9be9-5a786809d814 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:47.473835 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7b64f3-1142-4db3-9be9-5a786809d814 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:47.474099 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7b64f3-1142-4db3-9be9-5a786809d814 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:47.474823 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7b64f3-1142-4db3-9be9-5a786809d814 None admin] RBAC: Adding query filter params (role.id=c4bf019fc6c44c71a6c984f6d0bab75f, scope.project.id=7a48fb6b31f94e42b655fb380713f180, user.id=2cdd0cb043f94082854b2a11105255e6) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:18:47.475122 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7b64f3-1142-4db3-9be9-5a786809d814 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:18:47.476516 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7b64f3-1142-4db3-9be9-5a786809d814 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:18:47.482250 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 449/905] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Nov 2 07:18:47 2022] GET /identity/v3/role_assignments?user.id=2cdd0cb043f94082854b2a11105255e6&scope.project.id=7a48fb6b31f94e42b655fb380713f180&role.id=c4bf019fc6c44c71a6c984f6d0bab75f => generated 608 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:18:57.330208 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=62 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:18:57.479768 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=84 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:19:43.263519 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec288ab-ec60-4ec3-915b-7965b2722c76 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:19:43.263519 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec288ab-ec60-4ec3-915b-7965b2722c76 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:19:43.263519 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec288ab-ec60-4ec3-915b-7965b2722c76 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:19:43.263519 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 457/906] 10.222.0.18 () {54 vars in 927 bytes} [Wed Nov 2 07:19:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:19:43.273539 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac1a91b-444a-40b7-ab50-3ea579b66f2b None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:19:43.273539 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac1a91b-444a-40b7-ab50-3ea579b66f2b None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:19:43.273539 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac1a91b-444a-40b7-ab50-3ea579b66f2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:19:43.312275 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-8ac1a91b-444a-40b7-ab50-3ea579b66f2b None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:19:43.313437 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-8ac1a91b-444a-40b7-ab50-3ea579b66f2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:19:43.421806 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 450/907] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Nov 2 07:19:43 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 07:19:43.729171 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec288ab-ec60-4ec3-915b-7965b2722c76 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:19:43.863590 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec288ab-ec60-4ec3-915b-7965b2722c76 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:19:43.907431 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5c0891-d1f5-4ab7-97b5-cebe38d06d37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:19:43.908558 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5c0891-d1f5-4ab7-97b5-cebe38d06d37 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:19:43.908794 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5c0891-d1f5-4ab7-97b5-cebe38d06d37 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:19:43.909122 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5c0891-d1f5-4ab7-97b5-cebe38d06d37 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:19:43.921417 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5c0891-d1f5-4ab7-97b5-cebe38d06d37 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:19:43.923041 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5c0891-d1f5-4ab7-97b5-cebe38d06d37 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:19:43.941498 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-3e5c0891-d1f5-4ab7-97b5-cebe38d06d37 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:19:43.941498 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:19:43.941498 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:19:43.941498 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:19:43.941498 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:19:43.941498 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:19:43.941498 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:19:43.941498 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:19:43.941498 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:19:43.941498 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:19:43.941498 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:19:43.941498 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:19:43.941498 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:19:43.941498 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:19:43.941498 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:19:43.941498 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:19:43.943721 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:19:43.943721 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:19:43.943721 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:19:43.943721 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:19:43.943721 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:19:43.943721 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:19:43.943721 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:19:43.943721 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:19:43.943721 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:19:43.943721 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:19:43.943721 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:19:43.943721 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:19:43.943721 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:19:43.943721 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:19:43.943721 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:19:43.943721 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:19:43.943721 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:19:43.944692 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:19:43.944692 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:19:43.944692 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:19:43.944692 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 458/908] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Nov 2 07:19:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 07:19:43.951969 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac1a91b-444a-40b7-ab50-3ea579b66f2b None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:19:44.082165 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac1a91b-444a-40b7-ab50-3ea579b66f2b None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:19:44.119360 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:19:44.126316 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9574cf-0274-48b5-9622-a4513c5cc133 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:19:44.126955 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9574cf-0274-48b5-9622-a4513c5cc133 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:19:44.127236 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9574cf-0274-48b5-9622-a4513c5cc133 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:19:44.127507 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9574cf-0274-48b5-9622-a4513c5cc133 None admin] PATH_INFO: `/v3/domains` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:19:44.128154 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9574cf-0274-48b5-9622-a4513c5cc133 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:19:44.128455 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9574cf-0274-48b5-9622-a4513c5cc133 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:19:44.130642 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9574cf-0274-48b5-9622-a4513c5cc133 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:19:44.146399 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 451/909] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Nov 2 07:19:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:19:44.149316 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5c0891-d1f5-4ab7-97b5-cebe38d06d37 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:19:44.221570 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:19:44.283108 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5c0891-d1f5-4ab7-97b5-cebe38d06d37 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:19:44.334935 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb86801-f9f5-4180-bd58-135d34d96654 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:19:44.335984 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb86801-f9f5-4180-bd58-135d34d96654 None admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:19:44.336420 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb86801-f9f5-4180-bd58-135d34d96654 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:19:44.336842 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb86801-f9f5-4180-bd58-135d34d96654 None admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:19:44.337689 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb86801-f9f5-4180-bd58-135d34d96654 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:19:44.339293 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb86801-f9f5-4180-bd58-135d34d96654 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:19:44.374425 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 459/910] 10.222.0.18 () {60 vars in 1132 bytes} [Wed Nov 2 07:19:44 2022] POST /identity/v3/users => generated 251 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:19:46.758134 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1b3388-9d36-4336-99f8-26e0ed431fe7 None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:19:46.758727 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1b3388-9d36-4336-99f8-26e0ed431fe7 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:19:46.758727 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1b3388-9d36-4336-99f8-26e0ed431fe7 None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:19:46.760047 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 452/911] 10.222.0.18 () {54 vars in 927 bytes} [Wed Nov 2 07:19:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:19:46.765692 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1859cff-81f5-42b7-9051-a58185a59b74 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:19:46.766112 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1859cff-81f5-42b7-9051-a58185a59b74 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:19:46.766310 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1859cff-81f5-42b7-9051-a58185a59b74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:19:46.785405 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-a1859cff-81f5-42b7-9051-a58185a59b74 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:19:46.787328 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-a1859cff-81f5-42b7-9051-a58185a59b74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:19:46.862794 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 460/912] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Nov 2 07:19:46 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 07:19:47.171586 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1b3388-9d36-4336-99f8-26e0ed431fe7 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:19:47.292795 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1b3388-9d36-4336-99f8-26e0ed431fe7 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:19:47.335679 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fcfff7-d926-48ba-9db2-4242e2f4ac6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:19:47.336326 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fcfff7-d926-48ba-9db2-4242e2f4ac6c None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:19:47.336587 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fcfff7-d926-48ba-9db2-4242e2f4ac6c None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:19:47.336849 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fcfff7-d926-48ba-9db2-4242e2f4ac6c None admin] PATH_INFO: `/v3/roles/service` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:19:47.349415 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fcfff7-d926-48ba-9db2-4242e2f4ac6c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:19:47.354536 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-35fcfff7-d926-48ba-9db2-4242e2f4ac6c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Nov 02 07:19:47.354536 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:19:47.354536 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:19:47.354536 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:19:47.354536 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:19:47.354536 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:19:47.354536 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:19:47.354536 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:19:47.354536 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:19:47.354536 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:19:47.354536 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:19:47.354536 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:19:47.354536 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:19:47.354536 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:19:47.354536 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:19:47.354536 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:19:47.355499 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:19:47.355499 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:19:47.355499 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:19:47.355499 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:19:47.355499 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:19:47.355499 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:19:47.355499 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:19:47.355499 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:19:47.355499 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:19:47.355499 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:19:47.355499 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:19:47.355499 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Nov 02 07:19:47.355499 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:19:47.355499 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:19:47.355499 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:19:47.355499 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:19:47.359539 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:19:47.359539 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:19:47.359539 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:19:47.359539 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:19:47.359539 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:19:47.359539 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:19:47.359539 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:19:47.359539 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:19:47.359539 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:19:47.359539 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:19:47.359539 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Nov 02 07:19:47.359539 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Nov 02 07:19:47.359539 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 453/913] 10.222.0.18 () {56 vars in 1102 bytes} [Wed Nov 2 07:19:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:19:47.361971 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1859cff-81f5-42b7-9051-a58185a59b74 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:19:47.407978 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:19:47.474060 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1859cff-81f5-42b7-9051-a58185a59b74 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:19:47.511293 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b094f827-ebf8-4bb7-bc27-0b80077d425e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:19:47.512535 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b094f827-ebf8-4bb7-bc27-0b80077d425e None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:19:47.512535 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b094f827-ebf8-4bb7-bc27-0b80077d425e None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:19:47.512535 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b094f827-ebf8-4bb7-bc27-0b80077d425e None admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:19:47.513182 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b094f827-ebf8-4bb7-bc27-0b80077d425e None admin] RBAC: Adding query filter params (name=service) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:19:47.513480 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b094f827-ebf8-4bb7-bc27-0b80077d425e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:19:47.514953 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b094f827-ebf8-4bb7-bc27-0b80077d425e None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:19:47.524135 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 461/914] 10.222.0.18 () {56 vars in 1103 bytes} [Wed Nov 2 07:19:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:19:47.530098 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fcfff7-d926-48ba-9db2-4242e2f4ac6c None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:19:47.590152 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:19:47.647867 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fcfff7-d926-48ba-9db2-4242e2f4ac6c None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:19:47.686522 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab5c4b7-e58f-425d-8641-72ff38b61a93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:19:47.687165 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab5c4b7-e58f-425d-8641-72ff38b61a93 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:19:47.687444 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab5c4b7-e58f-425d-8641-72ff38b61a93 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:19:47.687773 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab5c4b7-e58f-425d-8641-72ff38b61a93 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:19:47.700156 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab5c4b7-e58f-425d-8641-72ff38b61a93 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:19:47.701589 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab5c4b7-e58f-425d-8641-72ff38b61a93 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:19:47.719205 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-2ab5c4b7-e58f-425d-8641-72ff38b61a93 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:19:47.719205 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:19:47.719205 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:19:47.719205 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:19:47.719205 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:19:47.719205 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:19:47.719205 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:19:47.719205 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:19:47.719205 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:19:47.719205 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:19:47.719205 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:19:47.719205 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:19:47.719205 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:19:47.719205 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:19:47.719205 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:19:47.719205 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:19:47.720409 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:19:47.720409 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:19:47.720409 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:19:47.720409 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:19:47.720409 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:19:47.720409 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:19:47.720409 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:19:47.720409 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Nov 02 07:19:47.720409 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:19:47.720409 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:19:47.720409 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:19:47.720409 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:19:47.720409 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:19:47.720409 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:19:47.720409 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:19:47.720409 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:19:47.720409 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:19:47.724182 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:19:47.724182 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:19:47.724182 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Nov 02 07:19:47.724182 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 454/915] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Nov 2 07:19:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:19:47.727738 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b094f827-ebf8-4bb7-bc27-0b80077d425e None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:19:47.834750 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:19:47.837967 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b094f827-ebf8-4bb7-bc27-0b80077d425e None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:19:47.881238 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba783d3-e065-4ff8-ac2b-c390bc3339af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:19:47.882137 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba783d3-e065-4ff8-ac2b-c390bc3339af None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:19:47.882450 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba783d3-e065-4ff8-ac2b-c390bc3339af None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:19:47.882723 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba783d3-e065-4ff8-ac2b-c390bc3339af None admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:19:47.883359 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba783d3-e065-4ff8-ac2b-c390bc3339af None admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:19:47.883657 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba783d3-e065-4ff8-ac2b-c390bc3339af None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:19:47.885012 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba783d3-e065-4ff8-ac2b-c390bc3339af None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:19:47.901986 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 462/916] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Nov 2 07:19:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:19:47.908943 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab5c4b7-e58f-425d-8641-72ff38b61a93 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:19:48.028920 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab5c4b7-e58f-425d-8641-72ff38b61a93 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:19:48.030639 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:19:48.071208 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e135393-78bc-47ed-960d-e50ca3090ea8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:19:48.072022 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e135393-78bc-47ed-960d-e50ca3090ea8 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:19:48.072383 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e135393-78bc-47ed-960d-e50ca3090ea8 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:19:48.072745 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e135393-78bc-47ed-960d-e50ca3090ea8 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:19:48.081333 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e135393-78bc-47ed-960d-e50ca3090ea8 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:19:48.083171 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e135393-78bc-47ed-960d-e50ca3090ea8 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:19:48.094836 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-8e135393-78bc-47ed-960d-e50ca3090ea8 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Nov 02 07:19:48.094836 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:19:48.094836 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:19:48.094836 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:19:48.094836 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:19:48.094836 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:19:48.094836 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:19:48.094836 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:19:48.094836 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:19:48.094836 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:19:48.094836 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:19:48.094836 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:19:48.094836 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Nov 02 07:19:48.094836 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_user(user_id) Nov 02 07:19:48.094836 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Nov 02 07:19:48.094836 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Nov 02 07:19:48.096113 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:19:48.096113 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:19:48.096113 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Nov 02 07:19:48.096113 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:19:48.096113 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Nov 02 07:19:48.096113 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:19:48.096113 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:19:48.096113 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:19:48.096113 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:19:48.096113 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:19:48.096113 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:19:48.096113 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Nov 02 07:19:48.096113 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:19:48.096113 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:19:48.096113 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:19:48.096113 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:19:48.096113 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:19:48.096113 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:19:48.097260 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:19:48.097260 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:19:48.097260 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Nov 02 07:19:48.097260 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Nov 02 07:19:48.097260 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Nov 02 07:19:48.097260 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Nov 02 07:19:48.097260 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Nov 02 07:19:48.097260 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Nov 02 07:19:48.097260 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Nov 02 07:19:48.097260 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Nov 02 07:19:48.100747 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 455/917] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Nov 2 07:19:47 2022] GET /identity/v3/users/placement => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 07:19:48.102008 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba783d3-e065-4ff8-ac2b-c390bc3339af None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:19:48.215007 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba783d3-e065-4ff8-ac2b-c390bc3339af None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:19:48.255175 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079cbe03-f407-4d4f-9e34-27a6e8fd54b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:19:48.255939 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079cbe03-f407-4d4f-9e34-27a6e8fd54b4 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:19:48.256222 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079cbe03-f407-4d4f-9e34-27a6e8fd54b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:19:48.256647 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079cbe03-f407-4d4f-9e34-27a6e8fd54b4 None admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:19:48.257467 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079cbe03-f407-4d4f-9e34-27a6e8fd54b4 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:19:48.257908 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079cbe03-f407-4d4f-9e34-27a6e8fd54b4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:19:48.259520 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079cbe03-f407-4d4f-9e34-27a6e8fd54b4 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:19:48.282077 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 463/918] 10.222.0.18 () {56 vars in 1143 bytes} [Wed Nov 2 07:19:48 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:19:48.287009 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e135393-78bc-47ed-960d-e50ca3090ea8 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:19:48.402874 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e135393-78bc-47ed-960d-e50ca3090ea8 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:19:48.439831 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be0bb94-0fa3-4508-bef4-bd4b43422e08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:19:48.440461 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be0bb94-0fa3-4508-bef4-bd4b43422e08 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:19:48.440735 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be0bb94-0fa3-4508-bef4-bd4b43422e08 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:19:48.441007 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be0bb94-0fa3-4508-bef4-bd4b43422e08 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:19:48.452008 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be0bb94-0fa3-4508-bef4-bd4b43422e08 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:19:48.453435 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be0bb94-0fa3-4508-bef4-bd4b43422e08 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:19:48.467675 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-8be0bb94-0fa3-4508-bef4-bd4b43422e08 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:19:48.467675 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:19:48.467675 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:19:48.467675 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:19:48.467675 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:19:48.467675 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:19:48.467675 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:19:48.467675 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:19:48.467675 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:19:48.467675 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:19:48.467675 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:19:48.467675 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:19:48.467675 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:19:48.467675 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:19:48.467675 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:19:48.467675 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:19:48.468806 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:19:48.468806 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:19:48.468806 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:19:48.468806 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:19:48.468806 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:19:48.468806 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:19:48.468806 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:19:48.468806 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Nov 02 07:19:48.468806 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:19:48.468806 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:19:48.468806 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:19:48.468806 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:19:48.468806 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:19:48.468806 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:19:48.468806 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:19:48.468806 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:19:48.468806 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:19:48.475110 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:19:48.475110 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:19:48.475110 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Nov 02 07:19:48.475110 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 456/919] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Nov 2 07:19:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:19:48.475406 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079cbe03-f407-4d4f-9e34-27a6e8fd54b4 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:19:48.494980 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:19:48.587017 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079cbe03-f407-4d4f-9e34-27a6e8fd54b4 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:19:48.623572 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0a8d08-dfdb-4959-8c7d-6e2dedea11b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:19:48.624172 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0a8d08-dfdb-4959-8c7d-6e2dedea11b2 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:19:48.624447 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0a8d08-dfdb-4959-8c7d-6e2dedea11b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:19:48.624711 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0a8d08-dfdb-4959-8c7d-6e2dedea11b2 None admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:19:48.625335 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0a8d08-dfdb-4959-8c7d-6e2dedea11b2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:19:48.625629 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0a8d08-dfdb-4959-8c7d-6e2dedea11b2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:19:48.627002 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0a8d08-dfdb-4959-8c7d-6e2dedea11b2 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:19:48.643540 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 464/920] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Nov 2 07:19:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:19:48.646344 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be0bb94-0fa3-4508-bef4-bd4b43422e08 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:19:48.668035 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:19:48.768633 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be0bb94-0fa3-4508-bef4-bd4b43422e08 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:19:48.808873 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0289536-7b3e-4935-95ff-c27787595075 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:19:48.809660 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0289536-7b3e-4935-95ff-c27787595075 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:19:48.810012 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0289536-7b3e-4935-95ff-c27787595075 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:19:48.810477 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0289536-7b3e-4935-95ff-c27787595075 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:19:48.818577 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0289536-7b3e-4935-95ff-c27787595075 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:19:48.820979 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0289536-7b3e-4935-95ff-c27787595075 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:19:48.841274 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-b0289536-7b3e-4935-95ff-c27787595075 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 07:19:48.841274 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:19:48.841274 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:19:48.841274 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:19:48.841274 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:19:48.841274 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:19:48.841274 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:19:48.841274 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:19:48.841274 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:19:48.841274 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:19:48.841274 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:19:48.841274 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:19:48.841274 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Nov 02 07:19:48.841274 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_project(project_id) Nov 02 07:19:48.841274 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Nov 02 07:19:48.842162 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Nov 02 07:19:48.842162 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:19:48.842162 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:19:48.842162 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:19:48.842162 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:19:48.842162 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:19:48.842162 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:19:48.842162 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:19:48.842162 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Nov 02 07:19:48.842162 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:19:48.842162 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:19:48.842162 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:19:48.842162 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:19:48.842162 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:19:48.842162 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:19:48.842162 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:19:48.842162 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:19:48.843086 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Nov 02 07:19:48.843086 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Nov 02 07:19:48.843086 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Nov 02 07:19:48.843086 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Nov 02 07:19:48.843086 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Nov 02 07:19:48.843086 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Nov 02 07:19:48.843086 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 07:19:48.843086 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Nov 02 07:19:48.846665 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 457/921] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Nov 2 07:19:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:19:48.852898 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0a8d08-dfdb-4959-8c7d-6e2dedea11b2 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:19:48.984096 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:19:49.008400 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0a8d08-dfdb-4959-8c7d-6e2dedea11b2 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:19:49.048071 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae22a5a-f50d-4cf0-b6f2-e49d9c1eb2b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:19:49.048679 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae22a5a-f50d-4cf0-b6f2-e49d9c1eb2b5 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:19:49.048957 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae22a5a-f50d-4cf0-b6f2-e49d9c1eb2b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:19:49.050081 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae22a5a-f50d-4cf0-b6f2-e49d9c1eb2b5 None admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:19:49.050865 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae22a5a-f50d-4cf0-b6f2-e49d9c1eb2b5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:19:49.051294 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae22a5a-f50d-4cf0-b6f2-e49d9c1eb2b5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:19:49.053083 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae22a5a-f50d-4cf0-b6f2-e49d9c1eb2b5 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:19:49.070534 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 465/922] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Nov 2 07:19:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:19:49.073211 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0289536-7b3e-4935-95ff-c27787595075 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:19:49.167355 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:19:49.196135 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0289536-7b3e-4935-95ff-c27787595075 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:19:49.233659 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3440e04-f06d-4a8f-b861-6ac71e7fb63a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:19:49.234318 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3440e04-f06d-4a8f-b861-6ac71e7fb63a None admin] REQUEST_METHOD: `PUT` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:19:49.234689 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3440e04-f06d-4a8f-b861-6ac71e7fb63a None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:19:49.234965 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3440e04-f06d-4a8f-b861-6ac71e7fb63a None admin] PATH_INFO: `/v3/projects/af04daa468b64160a6176dcab29502a7/users/4f2fc6a156c54d0c9d325901d0f7f6fe/roles/4af6f4c514284b8cbddf3e02ec826378` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:19:49.252131 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3440e04-f06d-4a8f-b861-6ac71e7fb63a None admin] RBAC: Authorizing `identity:create_grant(project_id=af04daa468b64160a6176dcab29502a7, user_id=4f2fc6a156c54d0c9d325901d0f7f6fe, role_id=4af6f4c514284b8cbddf3e02ec826378)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:19:49.253714 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3440e04-f06d-4a8f-b861-6ac71e7fb63a None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:19:49.268323 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 458/923] 10.222.0.18 () {58 vars in 1439 bytes} [Wed Nov 2 07:19:49 2022] PUT /identity/v3/projects/af04daa468b64160a6176dcab29502a7/users/4f2fc6a156c54d0c9d325901d0f7f6fe/roles/4af6f4c514284b8cbddf3e02ec826378 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 07:19:51.632746 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673fad37-f72c-494e-a2c9-336db71f39b1 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:19:51.633396 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673fad37-f72c-494e-a2c9-336db71f39b1 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:19:51.633396 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673fad37-f72c-494e-a2c9-336db71f39b1 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:19:51.634645 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 466/924] 10.222.0.18 () {54 vars in 927 bytes} [Wed Nov 2 07:19:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:19:51.643130 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da337106-7ff8-4e38-883b-3df07878569a None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:19:51.643130 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da337106-7ff8-4e38-883b-3df07878569a None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:19:51.643130 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da337106-7ff8-4e38-883b-3df07878569a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:19:51.661896 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-da337106-7ff8-4e38-883b-3df07878569a None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:19:51.663135 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-da337106-7ff8-4e38-883b-3df07878569a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:19:51.736878 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 459/925] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Nov 2 07:19:51 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 07:19:52.024684 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673fad37-f72c-494e-a2c9-336db71f39b1 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:19:52.132712 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673fad37-f72c-494e-a2c9-336db71f39b1 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:19:52.176833 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3458b91c-353e-4669-ab89-eee2896f2355 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:19:52.177444 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3458b91c-353e-4669-ab89-eee2896f2355 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:19:52.177719 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3458b91c-353e-4669-ab89-eee2896f2355 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:19:52.177986 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3458b91c-353e-4669-ab89-eee2896f2355 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:19:52.185576 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:19:52.191356 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3458b91c-353e-4669-ab89-eee2896f2355 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:19:52.197302 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-3458b91c-353e-4669-ab89-eee2896f2355 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Nov 02 07:19:52.197302 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:19:52.197302 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:19:52.197302 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:19:52.197302 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:19:52.197302 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:19:52.197302 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:19:52.197302 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:19:52.197302 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:19:52.197302 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:19:52.197302 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:19:52.197302 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:19:52.197302 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:19:52.197302 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:19:52.197302 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:19:52.197302 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:19:52.198233 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:19:52.198233 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:19:52.198233 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:19:52.198233 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:19:52.198233 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:19:52.198233 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:19:52.198233 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:19:52.198233 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:19:52.198233 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:19:52.198233 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:19:52.198233 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:19:52.198233 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:19:52.198233 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:19:52.198233 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:19:52.198233 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:19:52.198233 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:19:52.199280 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:19:52.199280 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:19:52.199280 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:19:52.199280 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:19:52.199280 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:19:52.199280 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:19:52.199280 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:19:52.199280 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:19:52.199280 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:19:52.199280 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:19:52.199280 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Nov 02 07:19:52.199280 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:19:52.199280 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 467/926] 10.222.0.18 () {56 vars in 1102 bytes} [Wed Nov 2 07:19:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:19:52.204379 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da337106-7ff8-4e38-883b-3df07878569a None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:19:52.261356 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:19:52.312880 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da337106-7ff8-4e38-883b-3df07878569a None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:19:52.349381 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01850fbf-4c27-4edd-899a-6d8933a24e97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:19:52.349990 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01850fbf-4c27-4edd-899a-6d8933a24e97 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:19:52.350378 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01850fbf-4c27-4edd-899a-6d8933a24e97 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:19:52.350581 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01850fbf-4c27-4edd-899a-6d8933a24e97 None admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:19:52.351234 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01850fbf-4c27-4edd-899a-6d8933a24e97 None admin] RBAC: Adding query filter params (name=service) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:19:52.351536 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01850fbf-4c27-4edd-899a-6d8933a24e97 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:19:52.352915 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01850fbf-4c27-4edd-899a-6d8933a24e97 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:19:52.361877 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 460/927] 10.222.0.18 () {56 vars in 1103 bytes} [Wed Nov 2 07:19:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:19:52.370059 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3458b91c-353e-4669-ab89-eee2896f2355 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:19:52.479317 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3458b91c-353e-4669-ab89-eee2896f2355 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:19:52.520052 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1793a4a-6b3b-4a02-b575-7360c2226855 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:19:52.521468 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1793a4a-6b3b-4a02-b575-7360c2226855 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:19:52.521750 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1793a4a-6b3b-4a02-b575-7360c2226855 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:19:52.522022 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1793a4a-6b3b-4a02-b575-7360c2226855 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:19:52.533572 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1793a4a-6b3b-4a02-b575-7360c2226855 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:19:52.535179 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1793a4a-6b3b-4a02-b575-7360c2226855 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:19:52.552341 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-c1793a4a-6b3b-4a02-b575-7360c2226855 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:19:52.552341 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:19:52.552341 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:19:52.552341 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:19:52.552341 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:19:52.552341 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:19:52.552341 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:19:52.552341 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:19:52.552341 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:19:52.552341 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:19:52.552341 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:19:52.552341 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:19:52.552341 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:19:52.552341 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:19:52.552341 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:19:52.552341 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:19:52.553542 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:19:52.553542 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:19:52.553542 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:19:52.553542 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:19:52.553542 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:19:52.553542 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:19:52.553542 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:19:52.553542 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:19:52.553542 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:19:52.553542 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:19:52.553542 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:19:52.553542 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:19:52.553542 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:19:52.553542 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:19:52.553542 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:19:52.553542 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:19:52.553542 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:19:52.554710 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:19:52.554710 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:19:52.554710 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:19:52.554710 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 468/928] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Nov 2 07:19:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:19:52.559795 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01850fbf-4c27-4edd-899a-6d8933a24e97 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:19:52.669622 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01850fbf-4c27-4edd-899a-6d8933a24e97 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:19:52.686745 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:19:52.708707 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7d0527-4d6a-4fbb-ae66-5af1e667f403 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:19:52.709255 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7d0527-4d6a-4fbb-ae66-5af1e667f403 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:19:52.709732 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7d0527-4d6a-4fbb-ae66-5af1e667f403 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:19:52.710021 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7d0527-4d6a-4fbb-ae66-5af1e667f403 None admin] PATH_INFO: `/v3/domains` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:19:52.710798 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7d0527-4d6a-4fbb-ae66-5af1e667f403 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:19:52.711071 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7d0527-4d6a-4fbb-ae66-5af1e667f403 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:19:52.713013 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7d0527-4d6a-4fbb-ae66-5af1e667f403 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:19:52.727016 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 461/929] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Nov 2 07:19:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:19:52.733378 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1793a4a-6b3b-4a02-b575-7360c2226855 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:19:52.798270 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:19:52.860186 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1793a4a-6b3b-4a02-b575-7360c2226855 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:19:52.898974 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f45929-dbfb-452a-8b8d-82521140d623 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:19:52.899530 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f45929-dbfb-452a-8b8d-82521140d623 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:19:52.899802 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f45929-dbfb-452a-8b8d-82521140d623 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:19:52.900073 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f45929-dbfb-452a-8b8d-82521140d623 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:19:52.908976 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f45929-dbfb-452a-8b8d-82521140d623 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:19:52.908976 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f45929-dbfb-452a-8b8d-82521140d623 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:19:52.922892 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-d4f45929-dbfb-452a-8b8d-82521140d623 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Nov 02 07:19:52.922892 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:19:52.922892 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:19:52.922892 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:19:52.922892 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:19:52.922892 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:19:52.922892 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:19:52.922892 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:19:52.922892 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:19:52.922892 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:19:52.922892 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:19:52.922892 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:19:52.922892 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Nov 02 07:19:52.922892 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_user(user_id) Nov 02 07:19:52.922892 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Nov 02 07:19:52.922892 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Nov 02 07:19:52.924045 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:19:52.924045 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:19:52.924045 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Nov 02 07:19:52.924045 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:19:52.924045 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Nov 02 07:19:52.924045 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:19:52.924045 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:19:52.924045 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:19:52.924045 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:19:52.924045 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:19:52.924045 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:19:52.924045 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:19:52.924045 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:19:52.924045 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:19:52.924045 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:19:52.924045 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:19:52.924045 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:19:52.924045 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:19:52.924976 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:19:52.924976 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:19:52.924976 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Nov 02 07:19:52.924976 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Nov 02 07:19:52.924976 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Nov 02 07:19:52.924976 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Nov 02 07:19:52.924976 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Nov 02 07:19:52.924976 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Nov 02 07:19:52.924976 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Nov 02 07:19:52.924976 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:19:52.924976 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 469/930] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Nov 2 07:19:52 2022] GET /identity/v3/users/placement => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:19:52.931325 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7d0527-4d6a-4fbb-ae66-5af1e667f403 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:19:53.038603 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7d0527-4d6a-4fbb-ae66-5af1e667f403 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:19:53.075726 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a18990a-cd1d-478f-a930-1eabf5392ea5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:19:53.076255 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a18990a-cd1d-478f-a930-1eabf5392ea5 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:19:53.076535 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a18990a-cd1d-478f-a930-1eabf5392ea5 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:19:53.076809 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a18990a-cd1d-478f-a930-1eabf5392ea5 None admin] PATH_INFO: `/v3/users` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:19:53.077520 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a18990a-cd1d-478f-a930-1eabf5392ea5 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:19:53.077816 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a18990a-cd1d-478f-a930-1eabf5392ea5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:19:53.079266 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a18990a-cd1d-478f-a930-1eabf5392ea5 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:19:53.097279 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 462/931] 10.222.0.18 () {56 vars in 1143 bytes} [Wed Nov 2 07:19:52 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:19:53.103578 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f45929-dbfb-452a-8b8d-82521140d623 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:19:53.215630 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f45929-dbfb-452a-8b8d-82521140d623 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:19:53.227995 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:19:53.253824 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4394ae-daed-46a0-a751-59de4920e704 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:19:53.254496 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4394ae-daed-46a0-a751-59de4920e704 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:19:53.255278 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4394ae-daed-46a0-a751-59de4920e704 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:19:53.255544 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4394ae-daed-46a0-a751-59de4920e704 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:19:53.322981 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4394ae-daed-46a0-a751-59de4920e704 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:19:53.324449 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4394ae-daed-46a0-a751-59de4920e704 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:19:53.440581 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 470/932] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Nov 2 07:19:53 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Nov 02 07:19:53.447382 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a18990a-cd1d-478f-a930-1eabf5392ea5 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:19:53.484524 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:19:53.557881 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a18990a-cd1d-478f-a930-1eabf5392ea5 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:19:53.595809 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b746af44-17b3-454c-970f-caf038d9f552 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:19:53.596433 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b746af44-17b3-454c-970f-caf038d9f552 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:19:53.596707 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b746af44-17b3-454c-970f-caf038d9f552 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:19:53.596975 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b746af44-17b3-454c-970f-caf038d9f552 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:19:53.608164 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b746af44-17b3-454c-970f-caf038d9f552 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:19:53.609761 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b746af44-17b3-454c-970f-caf038d9f552 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:19:53.625636 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-b746af44-17b3-454c-970f-caf038d9f552 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:19:53.625636 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:19:53.625636 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:19:53.625636 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:19:53.625636 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:19:53.625636 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:19:53.625636 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:19:53.625636 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:19:53.625636 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:19:53.625636 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:19:53.625636 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:19:53.625636 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:19:53.625636 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:19:53.625636 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:19:53.625636 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:19:53.625636 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:19:53.626695 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:19:53.626695 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:19:53.626695 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:19:53.626695 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:19:53.626695 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:19:53.626695 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:19:53.626695 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:19:53.626695 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Nov 02 07:19:53.626695 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:19:53.626695 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:19:53.626695 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:19:53.626695 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:19:53.626695 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:19:53.626695 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:19:53.626695 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:19:53.626695 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:19:53.626695 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:19:53.627921 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:19:53.627921 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:19:53.627921 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Nov 02 07:19:53.627921 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 463/933] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Nov 2 07:19:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:19:53.633717 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4394ae-daed-46a0-a751-59de4920e704 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:19:53.648067 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:19:53.742035 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4394ae-daed-46a0-a751-59de4920e704 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:19:53.777882 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d8bda9-9214-4ae3-8c98-f35bab37e516 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:19:53.778547 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d8bda9-9214-4ae3-8c98-f35bab37e516 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:19:53.778826 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d8bda9-9214-4ae3-8c98-f35bab37e516 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:19:53.779097 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d8bda9-9214-4ae3-8c98-f35bab37e516 None admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:19:53.779735 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d8bda9-9214-4ae3-8c98-f35bab37e516 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:19:53.780028 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d8bda9-9214-4ae3-8c98-f35bab37e516 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:19:53.781375 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d8bda9-9214-4ae3-8c98-f35bab37e516 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:19:53.792791 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 471/934] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Nov 2 07:19:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:19:53.798459 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b746af44-17b3-454c-970f-caf038d9f552 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:19:53.896194 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:19:53.915588 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b746af44-17b3-454c-970f-caf038d9f552 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:19:53.955178 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4775442f-bdf3-40c7-a9b5-2c26645c52ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:19:53.955901 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4775442f-bdf3-40c7-a9b5-2c26645c52ea None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:19:53.956317 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4775442f-bdf3-40c7-a9b5-2c26645c52ea None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:19:53.956650 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4775442f-bdf3-40c7-a9b5-2c26645c52ea None admin] PATH_INFO: `/v3/projects/service` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:19:53.963920 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4775442f-bdf3-40c7-a9b5-2c26645c52ea None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:19:53.965360 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4775442f-bdf3-40c7-a9b5-2c26645c52ea None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:19:53.977960 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-4775442f-bdf3-40c7-a9b5-2c26645c52ea None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 07:19:53.977960 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:19:53.977960 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:19:53.977960 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:19:53.977960 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:19:53.977960 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:19:53.977960 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:19:53.977960 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:19:53.977960 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:19:53.977960 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:19:53.977960 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:19:53.977960 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:19:53.977960 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Nov 02 07:19:53.977960 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_project(project_id) Nov 02 07:19:53.977960 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Nov 02 07:19:53.979073 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Nov 02 07:19:53.979073 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:19:53.979073 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:19:53.979073 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:19:53.979073 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:19:53.979073 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:19:53.979073 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:19:53.979073 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:19:53.979073 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Nov 02 07:19:53.979073 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:19:53.979073 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:19:53.979073 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:19:53.979073 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:19:53.979073 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:19:53.979073 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:19:53.979073 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:19:53.979073 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:19:53.979943 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Nov 02 07:19:53.979943 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Nov 02 07:19:53.979943 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Nov 02 07:19:53.979943 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Nov 02 07:19:53.979943 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Nov 02 07:19:53.979943 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Nov 02 07:19:53.979943 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 07:19:53.979943 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Nov 02 07:19:53.979943 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 464/935] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Nov 2 07:19:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 07:19:53.985650 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d8bda9-9214-4ae3-8c98-f35bab37e516 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:19:54.063888 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:19:54.100544 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d8bda9-9214-4ae3-8c98-f35bab37e516 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:19:54.141075 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338959c2-2698-4915-9de4-b93649954e8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:19:54.141564 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338959c2-2698-4915-9de4-b93649954e8b None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:19:54.141845 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338959c2-2698-4915-9de4-b93649954e8b None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:19:54.142111 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338959c2-2698-4915-9de4-b93649954e8b None admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:19:54.142856 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338959c2-2698-4915-9de4-b93649954e8b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:19:54.143136 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338959c2-2698-4915-9de4-b93649954e8b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:19:54.144842 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338959c2-2698-4915-9de4-b93649954e8b None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:19:54.157460 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 472/936] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Nov 2 07:19:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:19:54.163982 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4775442f-bdf3-40c7-a9b5-2c26645c52ea None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:19:54.270825 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4775442f-bdf3-40c7-a9b5-2c26645c52ea None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:19:54.305559 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acef53ae-3ca4-49c5-a232-e40bfdc9d529 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:19:54.306164 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acef53ae-3ca4-49c5-a232-e40bfdc9d529 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:19:54.306485 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acef53ae-3ca4-49c5-a232-e40bfdc9d529 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:19:54.306769 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acef53ae-3ca4-49c5-a232-e40bfdc9d529 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:19:54.307484 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acef53ae-3ca4-49c5-a232-e40bfdc9d529 None admin] RBAC: Adding query filter params (role.id=4af6f4c514284b8cbddf3e02ec826378, scope.project.id=af04daa468b64160a6176dcab29502a7, user.id=4f2fc6a156c54d0c9d325901d0f7f6fe) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:19:54.307791 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acef53ae-3ca4-49c5-a232-e40bfdc9d529 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:19:54.309202 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acef53ae-3ca4-49c5-a232-e40bfdc9d529 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:19:54.315846 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 465/937] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Nov 2 07:19:54 2022] GET /identity/v3/role_assignments?user.id=4f2fc6a156c54d0c9d325901d0f7f6fe&scope.project.id=af04daa468b64160a6176dcab29502a7&role.id=4af6f4c514284b8cbddf3e02ec826378 => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:19:56.508717 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615feffa-7a56-4645-871b-3c2ae98b26ab None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:19:56.509296 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615feffa-7a56-4645-871b-3c2ae98b26ab None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:19:56.509296 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615feffa-7a56-4645-871b-3c2ae98b26ab None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:19:56.510627 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 473/938] 10.222.0.18 () {54 vars in 927 bytes} [Wed Nov 2 07:19:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:19:56.516259 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7eab74f-1d59-47e8-860d-43420a5a9d94 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:19:56.516751 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7eab74f-1d59-47e8-860d-43420a5a9d94 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:19:56.516875 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7eab74f-1d59-47e8-860d-43420a5a9d94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:19:56.535498 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-d7eab74f-1d59-47e8-860d-43420a5a9d94 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:19:56.537268 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-d7eab74f-1d59-47e8-860d-43420a5a9d94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:19:56.540231 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:19:56.609913 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 466/939] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Nov 2 07:19:56 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 07:19:56.921567 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615feffa-7a56-4645-871b-3c2ae98b26ab None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:19:57.030234 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615feffa-7a56-4645-871b-3c2ae98b26ab None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:19:57.062784 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:19:57.070218 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee7af9c-d837-4866-beb3-c67e58b5caf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:19:57.071107 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee7af9c-d837-4866-beb3-c67e58b5caf8 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:19:57.071439 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee7af9c-d837-4866-beb3-c67e58b5caf8 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:19:57.071871 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee7af9c-d837-4866-beb3-c67e58b5caf8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:19:57.083900 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee7af9c-d837-4866-beb3-c67e58b5caf8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:19:57.088957 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-cee7af9c-d837-4866-beb3-c67e58b5caf8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 07:19:57.088957 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:19:57.088957 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:19:57.088957 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:19:57.088957 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:19:57.088957 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:19:57.088957 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:19:57.088957 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:19:57.088957 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:19:57.088957 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:19:57.088957 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:19:57.088957 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:19:57.088957 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:19:57.088957 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:19:57.088957 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:19:57.088957 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:19:57.090060 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:19:57.090060 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:19:57.090060 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:19:57.090060 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:19:57.090060 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:19:57.090060 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:19:57.090060 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:19:57.090060 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:19:57.090060 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:19:57.090060 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:19:57.090060 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:19:57.090060 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:19:57.090060 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:19:57.090060 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:19:57.090060 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:19:57.090060 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:19:57.091003 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:19:57.091003 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:19:57.091003 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:19:57.091003 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:19:57.091003 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:19:57.091003 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:19:57.091003 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:19:57.091003 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:19:57.091003 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:19:57.091003 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:19:57.091003 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 07:19:57.091003 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:19:57.091003 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 474/940] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Nov 2 07:19:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:19:57.096218 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7eab74f-1d59-47e8-860d-43420a5a9d94 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:19:57.211614 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7eab74f-1d59-47e8-860d-43420a5a9d94 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:19:57.252352 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65c7d1e-a1ae-426f-bb42-ca3c028c34f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:19:57.253149 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65c7d1e-a1ae-426f-bb42-ca3c028c34f6 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:19:57.253432 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65c7d1e-a1ae-426f-bb42-ca3c028c34f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:19:57.253905 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65c7d1e-a1ae-426f-bb42-ca3c028c34f6 None admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:19:57.254812 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65c7d1e-a1ae-426f-bb42-ca3c028c34f6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:19:57.255118 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65c7d1e-a1ae-426f-bb42-ca3c028c34f6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:19:57.256510 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65c7d1e-a1ae-426f-bb42-ca3c028c34f6 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:19:57.265623 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 467/941] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Nov 2 07:19:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:19:57.272186 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee7af9c-d837-4866-beb3-c67e58b5caf8 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:19:57.387593 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee7af9c-d837-4866-beb3-c67e58b5caf8 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:19:57.432225 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd19e35-7383-4e29-9610-8abb605c41ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:19:57.432851 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd19e35-7383-4e29-9610-8abb605c41ca None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:19:57.433126 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd19e35-7383-4e29-9610-8abb605c41ca None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:19:57.433393 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd19e35-7383-4e29-9610-8abb605c41ca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:19:57.444029 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd19e35-7383-4e29-9610-8abb605c41ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:19:57.445448 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd19e35-7383-4e29-9610-8abb605c41ca None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:19:57.460528 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-4dd19e35-7383-4e29-9610-8abb605c41ca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:19:57.460528 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:19:57.460528 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:19:57.460528 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:19:57.460528 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:19:57.460528 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:19:57.460528 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:19:57.460528 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:19:57.460528 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:19:57.460528 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:19:57.460528 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:19:57.460528 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:19:57.460528 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:19:57.460528 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:19:57.460528 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:19:57.460528 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:19:57.466523 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:19:57.466523 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:19:57.466523 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:19:57.466523 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:19:57.466523 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:19:57.466523 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:19:57.466523 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:19:57.466523 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:19:57.466523 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:19:57.466523 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:19:57.466523 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:19:57.466523 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:19:57.466523 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:19:57.466523 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:19:57.466523 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:19:57.466523 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:19:57.466523 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:19:57.467634 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:19:57.467634 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:19:57.467634 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:19:57.467634 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 475/942] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Nov 2 07:19:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:19:57.467918 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65c7d1e-a1ae-426f-bb42-ca3c028c34f6 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:19:57.470662 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:19:57.584060 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65c7d1e-a1ae-426f-bb42-ca3c028c34f6 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:19:57.618227 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04d5558-7da3-4704-af38-00a0a55e5a7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:19:57.618848 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04d5558-7da3-4704-af38-00a0a55e5a7e None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:19:57.619127 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04d5558-7da3-4704-af38-00a0a55e5a7e None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:19:57.619406 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04d5558-7da3-4704-af38-00a0a55e5a7e None admin] PATH_INFO: `/v3/domains` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:19:57.620037 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04d5558-7da3-4704-af38-00a0a55e5a7e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:19:57.620360 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04d5558-7da3-4704-af38-00a0a55e5a7e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:19:57.621702 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04d5558-7da3-4704-af38-00a0a55e5a7e None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:19:57.635219 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 468/943] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Nov 2 07:19:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:19:57.640975 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd19e35-7383-4e29-9610-8abb605c41ca None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:19:57.686672 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:19:57.747943 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd19e35-7383-4e29-9610-8abb605c41ca None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:19:57.782564 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500bd43d-c329-4ca7-8d4b-91796c0af852 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:19:57.783190 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500bd43d-c329-4ca7-8d4b-91796c0af852 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:19:57.783470 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500bd43d-c329-4ca7-8d4b-91796c0af852 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:19:57.783742 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500bd43d-c329-4ca7-8d4b-91796c0af852 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:19:57.791610 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500bd43d-c329-4ca7-8d4b-91796c0af852 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:19:57.793025 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500bd43d-c329-4ca7-8d4b-91796c0af852 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:19:57.804825 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-500bd43d-c329-4ca7-8d4b-91796c0af852 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Nov 02 07:19:57.804825 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:19:57.804825 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:19:57.804825 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:19:57.804825 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:19:57.804825 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:19:57.804825 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:19:57.804825 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:19:57.804825 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:19:57.804825 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:19:57.804825 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:19:57.804825 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:19:57.804825 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Nov 02 07:19:57.804825 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_user(user_id) Nov 02 07:19:57.804825 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Nov 02 07:19:57.804825 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Nov 02 07:19:57.805900 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:19:57.805900 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:19:57.805900 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Nov 02 07:19:57.805900 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:19:57.805900 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Nov 02 07:19:57.805900 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:19:57.805900 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:19:57.805900 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:19:57.805900 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:19:57.805900 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:19:57.805900 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:19:57.805900 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:19:57.805900 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:19:57.805900 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:19:57.805900 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:19:57.805900 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:19:57.805900 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:19:57.805900 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:19:57.807079 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:19:57.807079 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:19:57.807079 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Nov 02 07:19:57.807079 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Nov 02 07:19:57.807079 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Nov 02 07:19:57.807079 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Nov 02 07:19:57.807079 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Nov 02 07:19:57.807079 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Nov 02 07:19:57.807079 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Nov 02 07:19:57.807079 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:19:57.807079 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 476/944] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Nov 2 07:19:57 2022] GET /identity/v3/users/placement => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:19:57.813721 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04d5558-7da3-4704-af38-00a0a55e5a7e None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:19:57.883778 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:19:57.925161 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04d5558-7da3-4704-af38-00a0a55e5a7e None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:19:57.962501 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad8b892-5e72-4129-beb6-096aaa3054e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:19:57.963108 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad8b892-5e72-4129-beb6-096aaa3054e3 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:19:57.963390 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad8b892-5e72-4129-beb6-096aaa3054e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:19:57.963662 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad8b892-5e72-4129-beb6-096aaa3054e3 None admin] PATH_INFO: `/v3/users` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:19:57.964388 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad8b892-5e72-4129-beb6-096aaa3054e3 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:19:57.964691 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad8b892-5e72-4129-beb6-096aaa3054e3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:19:57.966101 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad8b892-5e72-4129-beb6-096aaa3054e3 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:19:57.984559 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 469/945] 10.222.0.18 () {56 vars in 1143 bytes} [Wed Nov 2 07:19:57 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:19:57.990900 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500bd43d-c329-4ca7-8d4b-91796c0af852 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:19:58.104082 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500bd43d-c329-4ca7-8d4b-91796c0af852 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:19:58.105882 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:19:58.143606 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50eff09-f36d-41fb-a8a2-08c2c40d3ca3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:19:58.144391 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50eff09-f36d-41fb-a8a2-08c2c40d3ca3 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:19:58.144768 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50eff09-f36d-41fb-a8a2-08c2c40d3ca3 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:19:58.145167 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50eff09-f36d-41fb-a8a2-08c2c40d3ca3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:19:58.156074 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50eff09-f36d-41fb-a8a2-08c2c40d3ca3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:19:58.157498 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50eff09-f36d-41fb-a8a2-08c2c40d3ca3 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:19:58.174755 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-a50eff09-f36d-41fb-a8a2-08c2c40d3ca3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:19:58.174755 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:19:58.174755 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:19:58.174755 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:19:58.174755 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:19:58.174755 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:19:58.174755 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:19:58.174755 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:19:58.174755 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:19:58.174755 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:19:58.174755 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:19:58.174755 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:19:58.174755 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:19:58.174755 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:19:58.174755 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:19:58.174755 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:19:58.176276 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:19:58.176276 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:19:58.176276 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:19:58.176276 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:19:58.176276 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:19:58.176276 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:19:58.176276 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:19:58.176276 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:19:58.176276 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:19:58.176276 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:19:58.176276 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:19:58.176276 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:19:58.176276 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:19:58.176276 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:19:58.176276 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:19:58.176276 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:19:58.176276 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:19:58.178015 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:19:58.178015 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:19:58.178015 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:19:58.178015 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 477/946] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Nov 2 07:19:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:19:58.182281 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad8b892-5e72-4129-beb6-096aaa3054e3 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:19:58.288143 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad8b892-5e72-4129-beb6-096aaa3054e3 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:19:58.310703 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:19:58.326580 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f630b6-5519-4145-86e2-58215aeaef2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:19:58.327148 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f630b6-5519-4145-86e2-58215aeaef2a None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:19:58.327483 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f630b6-5519-4145-86e2-58215aeaef2a None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:19:58.327752 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f630b6-5519-4145-86e2-58215aeaef2a None admin] PATH_INFO: `/v3/domains` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:19:58.328397 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f630b6-5519-4145-86e2-58215aeaef2a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:19:58.328765 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f630b6-5519-4145-86e2-58215aeaef2a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:19:58.330219 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f630b6-5519-4145-86e2-58215aeaef2a None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:19:58.342792 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 470/947] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Nov 2 07:19:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:19:58.349661 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50eff09-f36d-41fb-a8a2-08c2c40d3ca3 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:19:58.464185 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50eff09-f36d-41fb-a8a2-08c2c40d3ca3 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:19:58.499912 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becc757d-ece1-46cf-b57a-d236e0134b57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:19:58.500531 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becc757d-ece1-46cf-b57a-d236e0134b57 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:19:58.500801 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becc757d-ece1-46cf-b57a-d236e0134b57 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:19:58.501066 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becc757d-ece1-46cf-b57a-d236e0134b57 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:19:58.509028 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becc757d-ece1-46cf-b57a-d236e0134b57 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:19:58.510502 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becc757d-ece1-46cf-b57a-d236e0134b57 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:19:58.522536 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-becc757d-ece1-46cf-b57a-d236e0134b57 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 07:19:58.522536 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:19:58.522536 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:19:58.522536 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:19:58.522536 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:19:58.522536 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:19:58.522536 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:19:58.522536 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:19:58.522536 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:19:58.522536 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:19:58.522536 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:19:58.522536 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:19:58.522536 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Nov 02 07:19:58.522536 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_project(project_id) Nov 02 07:19:58.522536 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Nov 02 07:19:58.523708 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Nov 02 07:19:58.523708 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:19:58.523708 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:19:58.523708 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:19:58.523708 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:19:58.523708 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:19:58.523708 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:19:58.523708 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:19:58.523708 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:19:58.523708 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:19:58.523708 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:19:58.523708 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:19:58.523708 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:19:58.523708 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:19:58.523708 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:19:58.523708 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:19:58.523708 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:19:58.524636 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Nov 02 07:19:58.524636 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Nov 02 07:19:58.524636 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Nov 02 07:19:58.524636 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Nov 02 07:19:58.524636 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Nov 02 07:19:58.524636 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Nov 02 07:19:58.524636 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 07:19:58.524636 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:19:58.524636 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 478/948] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Nov 2 07:19:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 07:19:58.530092 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f630b6-5519-4145-86e2-58215aeaef2a None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:19:58.634092 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f630b6-5519-4145-86e2-58215aeaef2a None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:19:58.673749 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242e0bc1-2e11-45b6-ad46-b4d4e0a54795 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:19:58.674462 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242e0bc1-2e11-45b6-ad46-b4d4e0a54795 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:19:58.674800 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242e0bc1-2e11-45b6-ad46-b4d4e0a54795 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:19:58.675116 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242e0bc1-2e11-45b6-ad46-b4d4e0a54795 None admin] PATH_INFO: `/v3/projects` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:19:58.675875 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242e0bc1-2e11-45b6-ad46-b4d4e0a54795 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:19:58.676252 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242e0bc1-2e11-45b6-ad46-b4d4e0a54795 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:19:58.678407 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242e0bc1-2e11-45b6-ad46-b4d4e0a54795 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:19:58.691544 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 471/949] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Nov 2 07:19:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:19:58.698346 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becc757d-ece1-46cf-b57a-d236e0134b57 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:19:58.716671 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:19:58.807476 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becc757d-ece1-46cf-b57a-d236e0134b57 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:19:58.849092 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcee2b14-bf9c-4420-8177-a3a66373e3db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:19:58.849732 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcee2b14-bf9c-4420-8177-a3a66373e3db None admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:19:58.850008 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcee2b14-bf9c-4420-8177-a3a66373e3db None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:19:58.850271 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcee2b14-bf9c-4420-8177-a3a66373e3db None admin] PATH_INFO: `/v3/projects/af04daa468b64160a6176dcab29502a7/users/4f2fc6a156c54d0c9d325901d0f7f6fe/roles/c4bf019fc6c44c71a6c984f6d0bab75f` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:19:58.852199 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcee2b14-bf9c-4420-8177-a3a66373e3db None admin] RBAC: Authorizing `identity:create_grant(project_id=af04daa468b64160a6176dcab29502a7, user_id=4f2fc6a156c54d0c9d325901d0f7f6fe, role_id=c4bf019fc6c44c71a6c984f6d0bab75f)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:19:58.853601 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcee2b14-bf9c-4420-8177-a3a66373e3db None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:19:58.868338 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 479/950] 10.222.0.18 () {58 vars in 1439 bytes} [Wed Nov 2 07:19:58 2022] PUT /identity/v3/projects/af04daa468b64160a6176dcab29502a7/users/4f2fc6a156c54d0c9d325901d0f7f6fe/roles/c4bf019fc6c44c71a6c984f6d0bab75f => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:20:01.248129 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff81613f-7972-4497-be49-856def091dd9 None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:01.248666 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff81613f-7972-4497-be49-856def091dd9 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:01.248666 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff81613f-7972-4497-be49-856def091dd9 None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:01.250006 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 472/951] 10.222.0.18 () {54 vars in 927 bytes} [Wed Nov 2 07:20:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Nov 02 07:20:01.256412 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3424e08-c642-485d-9779-5db6c2964465 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:01.256857 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3424e08-c642-485d-9779-5db6c2964465 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:01.256967 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3424e08-c642-485d-9779-5db6c2964465 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:01.276087 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-c3424e08-c642-485d-9779-5db6c2964465 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:20:01.277296 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-c3424e08-c642-485d-9779-5db6c2964465 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:20:01.350809 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 480/952] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Nov 2 07:20:01 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 07:20:01.652241 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff81613f-7972-4497-be49-856def091dd9 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:20:01.699761 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:20:01.769093 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff81613f-7972-4497-be49-856def091dd9 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:20:01.807686 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6c766c-c14f-4845-a8c1-83589c2e0ecb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:20:01.808395 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6c766c-c14f-4845-a8c1-83589c2e0ecb None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:01.808698 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6c766c-c14f-4845-a8c1-83589c2e0ecb None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:01.809016 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6c766c-c14f-4845-a8c1-83589c2e0ecb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:01.820885 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6c766c-c14f-4845-a8c1-83589c2e0ecb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:20:01.826012 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-ef6c766c-c14f-4845-a8c1-83589c2e0ecb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 07:20:01.826012 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:20:01.826012 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:20:01.826012 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:20:01.826012 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:20:01.826012 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:20:01.826012 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:20:01.826012 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:20:01.826012 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:20:01.826012 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:20:01.826012 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:20:01.826012 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:20:01.826012 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:20:01.826012 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:20:01.826012 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:20:01.826012 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:20:01.827177 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:20:01.827177 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:20:01.827177 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:20:01.827177 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:20:01.827177 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:20:01.827177 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:20:01.827177 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:20:01.827177 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:20:01.827177 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:20:01.827177 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:20:01.827177 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:20:01.827177 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Nov 02 07:20:01.827177 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:20:01.827177 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:20:01.827177 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:20:01.827177 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:20:01.828473 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:20:01.828473 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:20:01.828473 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:20:01.828473 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:20:01.828473 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:20:01.828473 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:20:01.828473 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:20:01.828473 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:20:01.828473 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:20:01.828473 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:20:01.828473 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 07:20:01.828473 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Nov 02 07:20:01.828473 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 473/953] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Nov 2 07:20:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:20:01.833841 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3424e08-c642-485d-9779-5db6c2964465 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:20:01.837560 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:20:01.942968 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3424e08-c642-485d-9779-5db6c2964465 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:20:01.976900 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803dce04-2032-4453-990d-c3cb1317d2d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:20:01.977727 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803dce04-2032-4453-990d-c3cb1317d2d4 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:01.978188 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803dce04-2032-4453-990d-c3cb1317d2d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:01.978601 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803dce04-2032-4453-990d-c3cb1317d2d4 None admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:01.979394 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803dce04-2032-4453-990d-c3cb1317d2d4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:20:01.979817 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803dce04-2032-4453-990d-c3cb1317d2d4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:20:01.981647 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803dce04-2032-4453-990d-c3cb1317d2d4 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:20:01.990690 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 481/954] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Nov 2 07:20:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:20:01.997395 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6c766c-c14f-4845-a8c1-83589c2e0ecb None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:20:02.099904 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:20:02.103096 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6c766c-c14f-4845-a8c1-83589c2e0ecb None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:20:02.141545 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc26755-14dd-4113-bf01-9fdb7df3f6ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:20:02.142338 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc26755-14dd-4113-bf01-9fdb7df3f6ba None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:02.142745 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc26755-14dd-4113-bf01-9fdb7df3f6ba None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:02.143113 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc26755-14dd-4113-bf01-9fdb7df3f6ba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:02.153977 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc26755-14dd-4113-bf01-9fdb7df3f6ba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:20:02.155447 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc26755-14dd-4113-bf01-9fdb7df3f6ba None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:20:02.170040 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-2cc26755-14dd-4113-bf01-9fdb7df3f6ba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:20:02.170040 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:20:02.170040 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:20:02.170040 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:20:02.170040 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:20:02.170040 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:20:02.170040 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:20:02.170040 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:20:02.170040 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:20:02.170040 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:20:02.170040 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:20:02.170040 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:20:02.170040 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:20:02.170040 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:20:02.170040 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:20:02.170040 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:20:02.171203 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:20:02.171203 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:20:02.171203 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:20:02.171203 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:20:02.171203 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:20:02.171203 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:20:02.171203 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:20:02.171203 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Nov 02 07:20:02.171203 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:20:02.171203 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:20:02.171203 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:20:02.171203 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:20:02.171203 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:20:02.171203 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:20:02.171203 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:20:02.171203 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:20:02.171203 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:20:02.172293 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:20:02.172293 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:20:02.172293 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Nov 02 07:20:02.172293 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 474/955] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Nov 2 07:20:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:20:02.177647 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803dce04-2032-4453-990d-c3cb1317d2d4 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:20:02.253206 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:20:02.292042 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803dce04-2032-4453-990d-c3cb1317d2d4 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:20:02.332165 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0ceca2-1a65-4122-9c09-280b0e2e7871 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:20:02.333526 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0ceca2-1a65-4122-9c09-280b0e2e7871 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:02.333858 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0ceca2-1a65-4122-9c09-280b0e2e7871 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:02.334158 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0ceca2-1a65-4122-9c09-280b0e2e7871 None admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:02.334956 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0ceca2-1a65-4122-9c09-280b0e2e7871 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:20:02.335254 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0ceca2-1a65-4122-9c09-280b0e2e7871 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:20:02.336595 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0ceca2-1a65-4122-9c09-280b0e2e7871 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:20:02.350454 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 482/956] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Nov 2 07:20:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:20:02.356097 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc26755-14dd-4113-bf01-9fdb7df3f6ba None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:20:02.477840 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc26755-14dd-4113-bf01-9fdb7df3f6ba None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:20:02.518081 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ccfe15-a2ee-4b59-b6ce-c9572b244657 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:20:02.518657 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ccfe15-a2ee-4b59-b6ce-c9572b244657 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:02.518943 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ccfe15-a2ee-4b59-b6ce-c9572b244657 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:02.519283 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ccfe15-a2ee-4b59-b6ce-c9572b244657 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:02.529047 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ccfe15-a2ee-4b59-b6ce-c9572b244657 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:20:02.530595 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ccfe15-a2ee-4b59-b6ce-c9572b244657 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:20:02.544366 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-a1ccfe15-a2ee-4b59-b6ce-c9572b244657 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Nov 02 07:20:02.544366 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:20:02.544366 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:20:02.544366 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:20:02.544366 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:20:02.544366 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:20:02.544366 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:20:02.544366 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:20:02.544366 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:20:02.544366 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:20:02.544366 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:20:02.544366 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:20:02.544366 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Nov 02 07:20:02.544366 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_user(user_id) Nov 02 07:20:02.544366 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Nov 02 07:20:02.544366 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Nov 02 07:20:02.545656 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:20:02.545656 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:20:02.545656 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Nov 02 07:20:02.545656 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:20:02.545656 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Nov 02 07:20:02.545656 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:20:02.545656 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:20:02.545656 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:20:02.545656 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:20:02.545656 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:20:02.545656 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:20:02.545656 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Nov 02 07:20:02.545656 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:20:02.545656 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:20:02.545656 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:20:02.545656 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:20:02.545656 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:20:02.545656 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:20:02.546864 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:20:02.546864 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:20:02.546864 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Nov 02 07:20:02.546864 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Nov 02 07:20:02.546864 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Nov 02 07:20:02.546864 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Nov 02 07:20:02.546864 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Nov 02 07:20:02.546864 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Nov 02 07:20:02.546864 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Nov 02 07:20:02.546864 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Nov 02 07:20:02.546864 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 475/957] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Nov 2 07:20:02 2022] GET /identity/v3/users/placement => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:20:02.551974 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0ceca2-1a65-4122-9c09-280b0e2e7871 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:20:02.659275 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0ceca2-1a65-4122-9c09-280b0e2e7871 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:20:02.688771 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:20:02.697913 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c47df53-47e1-49a8-94b3-31f4a60a768f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:20:02.698549 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c47df53-47e1-49a8-94b3-31f4a60a768f None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:02.698842 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c47df53-47e1-49a8-94b3-31f4a60a768f None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:02.699091 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c47df53-47e1-49a8-94b3-31f4a60a768f None admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:02.699807 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c47df53-47e1-49a8-94b3-31f4a60a768f None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:20:02.700102 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c47df53-47e1-49a8-94b3-31f4a60a768f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:20:02.701500 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c47df53-47e1-49a8-94b3-31f4a60a768f None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:20:02.719059 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 483/958] 10.222.0.18 () {56 vars in 1143 bytes} [Wed Nov 2 07:20:02 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:20:02.725579 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ccfe15-a2ee-4b59-b6ce-c9572b244657 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:20:02.743610 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:20:02.837479 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ccfe15-a2ee-4b59-b6ce-c9572b244657 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:20:02.875514 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a6efeb-9be0-43de-a16b-0bfd75904f8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:20:02.876288 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a6efeb-9be0-43de-a16b-0bfd75904f8e None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:02.876668 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a6efeb-9be0-43de-a16b-0bfd75904f8e None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:02.877043 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a6efeb-9be0-43de-a16b-0bfd75904f8e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:02.964067 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a6efeb-9be0-43de-a16b-0bfd75904f8e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:20:02.965549 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a6efeb-9be0-43de-a16b-0bfd75904f8e None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:20:02.981060 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:20:03.069864 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 476/959] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Nov 2 07:20:02 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Nov 02 07:20:03.075749 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c47df53-47e1-49a8-94b3-31f4a60a768f None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:20:03.180325 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c47df53-47e1-49a8-94b3-31f4a60a768f None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:20:03.215521 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51182d36-28a9-46ba-a178-1c5a5ba30985 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:20:03.216157 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51182d36-28a9-46ba-a178-1c5a5ba30985 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:03.216432 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51182d36-28a9-46ba-a178-1c5a5ba30985 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:03.216697 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51182d36-28a9-46ba-a178-1c5a5ba30985 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:03.228197 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51182d36-28a9-46ba-a178-1c5a5ba30985 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:20:03.229640 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51182d36-28a9-46ba-a178-1c5a5ba30985 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:20:03.245445 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-51182d36-28a9-46ba-a178-1c5a5ba30985 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:20:03.245445 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:20:03.245445 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:20:03.245445 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:20:03.245445 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:20:03.245445 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:20:03.245445 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:20:03.245445 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:20:03.245445 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:20:03.245445 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:20:03.245445 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:20:03.245445 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:20:03.245445 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:20:03.245445 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:20:03.245445 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:20:03.245445 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:20:03.246674 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:20:03.246674 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:20:03.246674 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:20:03.246674 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:20:03.246674 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:20:03.246674 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:20:03.246674 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:20:03.246674 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:20:03.246674 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:20:03.246674 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:20:03.246674 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:20:03.246674 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:20:03.246674 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:20:03.246674 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:20:03.246674 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:20:03.246674 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:20:03.246674 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:20:03.247776 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:20:03.247776 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:20:03.247776 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:20:03.250531 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 484/960] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Nov 2 07:20:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:20:03.252365 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a6efeb-9be0-43de-a16b-0bfd75904f8e None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:20:03.355282 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a6efeb-9be0-43de-a16b-0bfd75904f8e None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:20:03.371250 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:20:03.394455 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4126927a-b0ce-495e-9073-17491e7e9fee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:20:03.395068 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4126927a-b0ce-495e-9073-17491e7e9fee None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:03.395918 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4126927a-b0ce-495e-9073-17491e7e9fee None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:03.396187 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4126927a-b0ce-495e-9073-17491e7e9fee None admin] PATH_INFO: `/v3/domains` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:03.396816 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4126927a-b0ce-495e-9073-17491e7e9fee None admin] RBAC: Adding query filter params (name=Default) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:20:03.397116 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4126927a-b0ce-495e-9073-17491e7e9fee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:20:03.398531 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4126927a-b0ce-495e-9073-17491e7e9fee None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:20:03.410601 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 477/961] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Nov 2 07:20:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:20:03.416600 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51182d36-28a9-46ba-a178-1c5a5ba30985 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:20:03.458454 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:20:03.525873 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51182d36-28a9-46ba-a178-1c5a5ba30985 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:20:03.562292 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bea3836-966f-4b11-92ec-017aaf012216 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:20:03.562948 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bea3836-966f-4b11-92ec-017aaf012216 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:03.563223 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bea3836-966f-4b11-92ec-017aaf012216 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:03.563489 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bea3836-966f-4b11-92ec-017aaf012216 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:03.570124 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bea3836-966f-4b11-92ec-017aaf012216 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:20:03.571588 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bea3836-966f-4b11-92ec-017aaf012216 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:20:03.582666 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-1bea3836-966f-4b11-92ec-017aaf012216 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 07:20:03.582666 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:20:03.582666 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:20:03.582666 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:20:03.582666 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:20:03.582666 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:20:03.582666 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:20:03.582666 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:20:03.582666 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:20:03.582666 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:20:03.582666 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:20:03.582666 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:20:03.582666 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Nov 02 07:20:03.582666 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_project(project_id) Nov 02 07:20:03.582666 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Nov 02 07:20:03.583632 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Nov 02 07:20:03.583632 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:20:03.583632 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:20:03.583632 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:20:03.583632 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:20:03.583632 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:20:03.583632 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:20:03.583632 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:20:03.583632 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:20:03.583632 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:20:03.583632 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:20:03.583632 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:20:03.583632 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:20:03.583632 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:20:03.583632 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:20:03.583632 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:20:03.583632 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:20:03.584706 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Nov 02 07:20:03.584706 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Nov 02 07:20:03.584706 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Nov 02 07:20:03.584706 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Nov 02 07:20:03.584706 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Nov 02 07:20:03.584706 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Nov 02 07:20:03.584706 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 07:20:03.584706 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:20:03.586208 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 485/962] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Nov 2 07:20:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:20:03.593072 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4126927a-b0ce-495e-9073-17491e7e9fee None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:20:03.689934 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4126927a-b0ce-495e-9073-17491e7e9fee None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:20:03.721494 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f773fc-8279-402c-913a-2a3720212b99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:20:03.722071 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f773fc-8279-402c-913a-2a3720212b99 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:03.722351 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f773fc-8279-402c-913a-2a3720212b99 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:03.722645 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f773fc-8279-402c-913a-2a3720212b99 None admin] PATH_INFO: `/v3/projects` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:03.723851 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f773fc-8279-402c-913a-2a3720212b99 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:20:03.724145 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f773fc-8279-402c-913a-2a3720212b99 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:20:03.725546 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f773fc-8279-402c-913a-2a3720212b99 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:20:03.737138 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 478/963] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Nov 2 07:20:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:20:03.743266 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bea3836-966f-4b11-92ec-017aaf012216 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:20:03.834057 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:20:03.840142 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bea3836-966f-4b11-92ec-017aaf012216 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:20:03.872087 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bce40da-bda5-4157-8168-2e2f966c9e3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:20:03.872665 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bce40da-bda5-4157-8168-2e2f966c9e3b None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:03.872937 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bce40da-bda5-4157-8168-2e2f966c9e3b None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:03.873198 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bce40da-bda5-4157-8168-2e2f966c9e3b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:03.873900 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bce40da-bda5-4157-8168-2e2f966c9e3b None admin] RBAC: Adding query filter params (role.id=c4bf019fc6c44c71a6c984f6d0bab75f, scope.project.id=af04daa468b64160a6176dcab29502a7, user.id=4f2fc6a156c54d0c9d325901d0f7f6fe) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:20:03.874196 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bce40da-bda5-4157-8168-2e2f966c9e3b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:20:03.875611 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bce40da-bda5-4157-8168-2e2f966c9e3b None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:20:03.881015 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 486/964] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Nov 2 07:20:03 2022] GET /identity/v3/role_assignments?user.id=4f2fc6a156c54d0c9d325901d0f7f6fe&scope.project.id=af04daa468b64160a6176dcab29502a7&role.id=c4bf019fc6c44c71a6c984f6d0bab75f => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:20:06.033419 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9259f818-a0af-4f85-a330-6b33dcd5e784 None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:06.033998 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9259f818-a0af-4f85-a330-6b33dcd5e784 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:06.033998 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9259f818-a0af-4f85-a330-6b33dcd5e784 None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:06.035340 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 479/965] 10.222.0.18 () {54 vars in 927 bytes} [Wed Nov 2 07:20:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:20:06.040903 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0566e723-05bc-4af9-80ca-e28d3e36b67a None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:06.041302 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0566e723-05bc-4af9-80ca-e28d3e36b67a None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:06.041446 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0566e723-05bc-4af9-80ca-e28d3e36b67a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:06.059126 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-0566e723-05bc-4af9-80ca-e28d3e36b67a None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:20:06.060287 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-0566e723-05bc-4af9-80ca-e28d3e36b67a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:20:06.123135 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 487/966] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Nov 2 07:20:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 07:20:06.398160 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9259f818-a0af-4f85-a330-6b33dcd5e784 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:20:06.432805 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:20:06.502411 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9259f818-a0af-4f85-a330-6b33dcd5e784 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:20:06.534969 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff6d3e8-8322-4b9b-a748-25b0a267131f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:20:06.535585 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff6d3e8-8322-4b9b-a748-25b0a267131f None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:06.535912 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff6d3e8-8322-4b9b-a748-25b0a267131f None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:06.536177 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff6d3e8-8322-4b9b-a748-25b0a267131f None admin] PATH_INFO: `/v3/services/placement` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:06.536741 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff6d3e8-8322-4b9b-a748-25b0a267131f None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:20:06.538464 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff6d3e8-8322-4b9b-a748-25b0a267131f None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:20:06.546845 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-eff6d3e8-8322-4b9b-a748-25b0a267131f None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Nov 02 07:20:06.546845 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:20:06.546845 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Nov 02 07:20:06.546845 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Nov 02 07:20:06.546845 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Nov 02 07:20:06.546845 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Nov 02 07:20:06.546845 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Nov 02 07:20:06.546845 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:20:06.546845 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Nov 02 07:20:06.546845 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Nov 02 07:20:06.546845 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 07:20:06.546845 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Nov 02 07:20:06.546845 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:20:06.546845 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:20:06.546845 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:20:06.546845 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:20:06.546845 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:20:06.548100 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:20:06.548100 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:20:06.548100 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:20:06.548100 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:20:06.548100 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:20:06.548100 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:20:06.548100 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Nov 02 07:20:06.548100 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_service(service_id) Nov 02 07:20:06.548100 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Nov 02 07:20:06.548100 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Nov 02 07:20:06.548100 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:20:06.548100 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:20:06.548100 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:20:06.548100 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:20:06.548100 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:20:06.548100 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:20:06.549012 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:20:06.549012 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Nov 02 07:20:06.549012 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:20:06.549012 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:20:06.549012 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:20:06.549012 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:20:06.549012 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:20:06.549012 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:20:06.549012 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:20:06.549012 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:20:06.549012 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Nov 02 07:20:06.549012 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:20:06.549012 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Nov 02 07:20:06.549012 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Nov 02 07:20:06.549012 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 480/967] 10.222.0.18 () {56 vars in 1117 bytes} [Wed Nov 2 07:20:06 2022] GET /identity/v3/services/placement => generated 90 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:20:06.553942 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0566e723-05bc-4af9-80ca-e28d3e36b67a None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:20:06.639477 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:20:06.650465 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0566e723-05bc-4af9-80ca-e28d3e36b67a None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:20:06.683021 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4bf72d-5435-4c42-975d-39c3ea0c4734 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:20:06.683610 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4bf72d-5435-4c42-975d-39c3ea0c4734 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:06.683882 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4bf72d-5435-4c42-975d-39c3ea0c4734 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:06.684146 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4bf72d-5435-4c42-975d-39c3ea0c4734 None admin] PATH_INFO: `/v3/services` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:06.684757 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4bf72d-5435-4c42-975d-39c3ea0c4734 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:20:06.685050 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4bf72d-5435-4c42-975d-39c3ea0c4734 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:20:06.686407 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4bf72d-5435-4c42-975d-39c3ea0c4734 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:20:06.691003 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 488/968] 10.222.0.18 () {56 vars in 1116 bytes} [Wed Nov 2 07:20:06 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:20:06.697267 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff6d3e8-8322-4b9b-a748-25b0a267131f None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:20:06.788500 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:20:06.794496 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff6d3e8-8322-4b9b-a748-25b0a267131f None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:20:06.827297 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2ff151-940e-4ab3-846a-f470c342c66f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:20:06.827882 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2ff151-940e-4ab3-846a-f470c342c66f None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:06.828164 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2ff151-940e-4ab3-846a-f470c342c66f None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:06.828434 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2ff151-940e-4ab3-846a-f470c342c66f None admin] PATH_INFO: `/v3/services` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:06.829046 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2ff151-940e-4ab3-846a-f470c342c66f None admin] RBAC: Adding query filter params (type=placement) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:20:06.829343 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2ff151-940e-4ab3-846a-f470c342c66f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:20:06.830718 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2ff151-940e-4ab3-846a-f470c342c66f None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:20:06.835241 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 481/969] 10.222.0.18 () {56 vars in 1116 bytes} [Wed Nov 2 07:20:06 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:20:08.990446 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879e135e-be98-4cca-9df7-57471b6e120d None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:08.990972 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879e135e-be98-4cca-9df7-57471b6e120d None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:08.990972 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879e135e-be98-4cca-9df7-57471b6e120d None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:08.992211 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 489/970] 10.222.0.18 () {54 vars in 927 bytes} [Wed Nov 2 07:20:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:20:08.997271 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa77f8f-158c-4e32-963c-55aadc5e0daa None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:08.997702 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa77f8f-158c-4e32-963c-55aadc5e0daa None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:08.997800 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa77f8f-158c-4e32-963c-55aadc5e0daa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:09.015929 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-ffa77f8f-158c-4e32-963c-55aadc5e0daa None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:20:09.017089 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-ffa77f8f-158c-4e32-963c-55aadc5e0daa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:20:09.079812 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 482/971] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Nov 2 07:20:08 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 07:20:09.350537 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879e135e-be98-4cca-9df7-57471b6e120d None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:20:09.452786 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879e135e-be98-4cca-9df7-57471b6e120d None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:20:09.485208 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b9ee61-24ff-4d7c-adb8-1b9048d0c002 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:20:09.485794 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b9ee61-24ff-4d7c-adb8-1b9048d0c002 None admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:09.486069 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b9ee61-24ff-4d7c-adb8-1b9048d0c002 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:09.486336 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b9ee61-24ff-4d7c-adb8-1b9048d0c002 None admin] PATH_INFO: `/v3/services` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:09.487002 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b9ee61-24ff-4d7c-adb8-1b9048d0c002 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:20:09.488345 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b9ee61-24ff-4d7c-adb8-1b9048d0c002 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:20:09.502314 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 490/972] 10.222.0.18 () {60 vars in 1141 bytes} [Wed Nov 2 07:20:09 2022] POST /identity/v3/services => generated 246 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:20:11.661533 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79f89df-e3b9-40c5-824f-68333e9a7fc5 None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:11.662004 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79f89df-e3b9-40c5-824f-68333e9a7fc5 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:11.662105 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79f89df-e3b9-40c5-824f-68333e9a7fc5 None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:11.663466 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 483/973] 10.222.0.18 () {54 vars in 927 bytes} [Wed Nov 2 07:20:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:20:11.668479 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04876828-afae-4b24-91e3-2119b960cd11 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:11.668875 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04876828-afae-4b24-91e3-2119b960cd11 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:11.669017 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04876828-afae-4b24-91e3-2119b960cd11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:11.686497 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-04876828-afae-4b24-91e3-2119b960cd11 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:20:11.687659 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-04876828-afae-4b24-91e3-2119b960cd11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:20:11.697039 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:20:11.759559 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 491/974] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Nov 2 07:20:11 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 07:20:12.024290 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79f89df-e3b9-40c5-824f-68333e9a7fc5 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:20:12.112047 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:20:12.123577 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79f89df-e3b9-40c5-824f-68333e9a7fc5 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:20:12.156083 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb390c4-3dca-4405-a772-7bcec27c80bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:20:12.156692 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb390c4-3dca-4405-a772-7bcec27c80bc None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:12.156968 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb390c4-3dca-4405-a772-7bcec27c80bc None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:12.157237 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb390c4-3dca-4405-a772-7bcec27c80bc None admin] PATH_INFO: `/v3/services/placement` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:12.157803 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb390c4-3dca-4405-a772-7bcec27c80bc None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:20:12.159170 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb390c4-3dca-4405-a772-7bcec27c80bc None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:20:12.167527 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-7bb390c4-3dca-4405-a772-7bcec27c80bc None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Nov 02 07:20:12.167527 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:20:12.167527 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Nov 02 07:20:12.167527 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Nov 02 07:20:12.167527 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Nov 02 07:20:12.167527 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Nov 02 07:20:12.167527 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Nov 02 07:20:12.167527 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:20:12.167527 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Nov 02 07:20:12.167527 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Nov 02 07:20:12.167527 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 07:20:12.167527 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Nov 02 07:20:12.167527 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:20:12.167527 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:20:12.167527 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:20:12.167527 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:20:12.167527 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:20:12.168747 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:20:12.168747 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:20:12.168747 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:20:12.168747 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:20:12.168747 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:20:12.168747 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:20:12.168747 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Nov 02 07:20:12.168747 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_service(service_id) Nov 02 07:20:12.168747 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Nov 02 07:20:12.168747 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Nov 02 07:20:12.168747 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:20:12.168747 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:20:12.168747 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:20:12.168747 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:20:12.168747 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:20:12.168747 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:20:12.169881 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:20:12.169881 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Nov 02 07:20:12.169881 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:20:12.169881 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:20:12.169881 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:20:12.169881 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:20:12.169881 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:20:12.169881 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:20:12.169881 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:20:12.169881 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:20:12.169881 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Nov 02 07:20:12.169881 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:20:12.169881 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Nov 02 07:20:12.169881 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Nov 02 07:20:12.169881 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 484/975] 10.222.0.18 () {56 vars in 1117 bytes} [Wed Nov 2 07:20:12 2022] GET /identity/v3/services/placement => generated 90 bytes in 145 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:20:12.174416 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04876828-afae-4b24-91e3-2119b960cd11 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:20:12.270014 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04876828-afae-4b24-91e3-2119b960cd11 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:20:12.302780 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b42da2e-0536-405a-8e20-d65b459a6fc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:20:12.303304 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b42da2e-0536-405a-8e20-d65b459a6fc8 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:12.303581 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b42da2e-0536-405a-8e20-d65b459a6fc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:12.303848 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b42da2e-0536-405a-8e20-d65b459a6fc8 None admin] PATH_INFO: `/v3/services` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:12.304460 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b42da2e-0536-405a-8e20-d65b459a6fc8 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:20:12.304758 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b42da2e-0536-405a-8e20-d65b459a6fc8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:20:12.306094 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b42da2e-0536-405a-8e20-d65b459a6fc8 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:20:12.310996 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 492/976] 10.222.0.18 () {56 vars in 1116 bytes} [Wed Nov 2 07:20:12 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:20:12.316803 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb390c4-3dca-4405-a772-7bcec27c80bc None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:20:12.426919 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb390c4-3dca-4405-a772-7bcec27c80bc None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:20:12.462658 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ab9c96-cbcd-4604-8b9e-d622aea5b5ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:20:12.463253 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ab9c96-cbcd-4604-8b9e-d622aea5b5ac None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:12.463532 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ab9c96-cbcd-4604-8b9e-d622aea5b5ac None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:12.463800 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ab9c96-cbcd-4604-8b9e-d622aea5b5ac None admin] PATH_INFO: `/v3/endpoints` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:12.464481 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ab9c96-cbcd-4604-8b9e-d622aea5b5ac None admin] RBAC: Adding query filter params (interface=public, service_id=9e0249733bdd4bdfa8cdcf8b76ec3475, region_id=RegionOne) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:20:12.464786 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ab9c96-cbcd-4604-8b9e-d622aea5b5ac None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:20:12.466139 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ab9c96-cbcd-4604-8b9e-d622aea5b5ac None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:20:12.470951 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 485/977] 10.222.0.18 () {56 vars in 1251 bytes} [Wed Nov 2 07:20:12 2022] GET /identity/v3/endpoints?service_id=9e0249733bdd4bdfa8cdcf8b76ec3475&interface=public®ion_id=RegionOne => generated 194 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:20:12.476661 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b42da2e-0536-405a-8e20-d65b459a6fc8 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:20:12.480300 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:20:12.573207 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b42da2e-0536-405a-8e20-d65b459a6fc8 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:20:12.605185 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53b4627-846f-404c-95a9-408bdbf503e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:20:12.605768 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53b4627-846f-404c-95a9-408bdbf503e5 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:12.606043 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53b4627-846f-404c-95a9-408bdbf503e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:12.606311 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53b4627-846f-404c-95a9-408bdbf503e5 None admin] PATH_INFO: `/v3/services` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:12.607185 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53b4627-846f-404c-95a9-408bdbf503e5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:20:12.608511 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53b4627-846f-404c-95a9-408bdbf503e5 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:20:12.615072 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 493/978] 10.222.0.18 () {56 vars in 1087 bytes} [Wed Nov 2 07:20:12 2022] GET /identity/v3/services => generated 2191 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:20:14.748882 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f911f7b5-d171-4f80-8978-f591103f5b30 None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:14.749432 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f911f7b5-d171-4f80-8978-f591103f5b30 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:14.749432 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f911f7b5-d171-4f80-8978-f591103f5b30 None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:14.750792 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 486/979] 10.222.0.18 () {54 vars in 927 bytes} [Wed Nov 2 07:20:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:20:14.756404 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c76e56-04dc-466a-91c6-88e021913758 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:14.756775 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c76e56-04dc-466a-91c6-88e021913758 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:14.756942 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c76e56-04dc-466a-91c6-88e021913758 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:14.775123 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-a7c76e56-04dc-466a-91c6-88e021913758 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:20:14.776282 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-a7c76e56-04dc-466a-91c6-88e021913758 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:20:14.834057 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:20:14.839650 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 494/980] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Nov 2 07:20:14 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Nov 02 07:20:15.104724 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f911f7b5-d171-4f80-8978-f591103f5b30 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:20:15.124490 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:20:15.204110 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f911f7b5-d171-4f80-8978-f591103f5b30 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:20:15.236178 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7189019-9c44-434d-93ec-f4e1b29f3b66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:20:15.236796 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7189019-9c44-434d-93ec-f4e1b29f3b66 None admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:15.237073 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7189019-9c44-434d-93ec-f4e1b29f3b66 None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:15.237341 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7189019-9c44-434d-93ec-f4e1b29f3b66 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:15.237908 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7189019-9c44-434d-93ec-f4e1b29f3b66 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:20:15.239802 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7189019-9c44-434d-93ec-f4e1b29f3b66 None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:20:15.248264 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application [None req-c7189019-9c44-434d-93ec-f4e1b29f3b66 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Nov 02 07:20:15.248264 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:20:15.248264 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Nov 02 07:20:15.248264 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Nov 02 07:20:15.248264 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Nov 02 07:20:15.248264 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Nov 02 07:20:15.248264 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Nov 02 07:20:15.248264 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:20:15.248264 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Nov 02 07:20:15.248264 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Nov 02 07:20:15.248264 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 07:20:15.248264 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Nov 02 07:20:15.248264 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:20:15.248264 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:20:15.248264 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:20:15.248264 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:20:15.248264 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:20:15.249254 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:20:15.249254 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:20:15.249254 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:20:15.249254 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:20:15.249254 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:20:15.249254 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:20:15.249254 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Nov 02 07:20:15.249254 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._get_service(service_id) Nov 02 07:20:15.249254 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Nov 02 07:20:15.249254 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Nov 02 07:20:15.249254 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:20:15.249254 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:20:15.249254 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:20:15.249254 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:20:15.249254 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:20:15.249254 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:20:15.250075 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:20:15.250075 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application with Lock( Nov 02 07:20:15.250075 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:20:15.250075 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:20:15.250075 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:20:15.250075 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:20:15.250075 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:20:15.250075 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:20:15.250075 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:20:15.250075 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:20:15.250075 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Nov 02 07:20:15.250075 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:20:15.250075 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Nov 02 07:20:15.250075 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: ERROR keystone.server.flask.application Nov 02 07:20:15.250075 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 487/981] 10.222.0.18 () {56 vars in 1117 bytes} [Wed Nov 2 07:20:15 2022] GET /identity/v3/services/placement => generated 90 bytes in 145 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:20:15.255836 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c76e56-04dc-466a-91c6-88e021913758 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:20:15.349737 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c76e56-04dc-466a-91c6-88e021913758 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:20:15.391915 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f56c4c8-49d2-4ab6-8d7e-db2df62c9dd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:20:15.392539 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f56c4c8-49d2-4ab6-8d7e-db2df62c9dd5 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:15.392864 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f56c4c8-49d2-4ab6-8d7e-db2df62c9dd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:15.393227 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f56c4c8-49d2-4ab6-8d7e-db2df62c9dd5 None admin] PATH_INFO: `/v3/services` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:15.394257 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f56c4c8-49d2-4ab6-8d7e-db2df62c9dd5 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:20:15.394638 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f56c4c8-49d2-4ab6-8d7e-db2df62c9dd5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:20:15.396651 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f56c4c8-49d2-4ab6-8d7e-db2df62c9dd5 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:20:15.403508 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 495/982] 10.222.0.18 () {56 vars in 1116 bytes} [Wed Nov 2 07:20:15 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:20:15.410683 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7189019-9c44-434d-93ec-f4e1b29f3b66 None admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:20:15.498260 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:20:15.515904 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7189019-9c44-434d-93ec-f4e1b29f3b66 None admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:20:15.548322 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb110c72-17dc-4c76-9da2-3ee01935db4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:20:15.548916 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb110c72-17dc-4c76-9da2-3ee01935db4d None admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:15.549196 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb110c72-17dc-4c76-9da2-3ee01935db4d None admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:15.549462 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb110c72-17dc-4c76-9da2-3ee01935db4d None admin] PATH_INFO: `/v3/endpoints` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:15.550114 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb110c72-17dc-4c76-9da2-3ee01935db4d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:20:15.552179 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb110c72-17dc-4c76-9da2-3ee01935db4d None admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:20:15.570440 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 488/983] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Nov 2 07:20:15 2022] POST /identity/v3/endpoints => generated 331 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 07:20:25.399470 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=69 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:20:25.568764 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=30,keystone:INSERT=1 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:21:19.589619 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c8cb55-3a77-412d-a83b-d31fa76fff75 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:19.590117 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c8cb55-3a77-412d-a83b-d31fa76fff75 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:19.590344 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c8cb55-3a77-412d-a83b-d31fa76fff75 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:19.596166 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 496/984] 10.222.0.18 () {54 vars in 927 bytes} [Wed Nov 2 07:21:19 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:21:19.597639 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d1e1d1-fc10-41cd-af9b-fdf9e8fef004 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:19.598072 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d1e1d1-fc10-41cd-af9b-fdf9e8fef004 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:19.598241 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d1e1d1-fc10-41cd-af9b-fdf9e8fef004 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:19.742513 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-85d1e1d1-fc10-41cd-af9b-fdf9e8fef004 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:21:19.744841 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-85d1e1d1-fc10-41cd-af9b-fdf9e8fef004 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:21:19.896187 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 489/985] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Nov 2 07:21:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 299 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:21:20.197014 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94fbb20-7994-4ba4-b2ef-ed21b361f42d None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:20.197294 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94fbb20-7994-4ba4-b2ef-ed21b361f42d None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:20.197741 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94fbb20-7994-4ba4-b2ef-ed21b361f42d None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:20.199136 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 497/986] 10.222.0.18 () {54 vars in 965 bytes} [Wed Nov 2 07:21:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:21:20.205668 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d252b8-c55c-471b-925d-9c4c89a670b0 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:20.205991 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d252b8-c55c-471b-925d-9c4c89a670b0 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:20.206240 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d252b8-c55c-471b-925d-9c4c89a670b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:20.299665 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-14d252b8-c55c-471b-925d-9c4c89a670b0 None None] MFA Rules not processed for user `b3ec6ecb85f94ef59762877d9ea33ed0`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:21:20.301046 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-14d252b8-c55c-471b-925d-9c4c89a670b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:21:20.326012 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:21:20.389139 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 490/987] 10.222.0.18 () {58 vars in 1062 bytes} [Wed Nov 2 07:21:20 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:21:20.399040 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94fbb20-7994-4ba4-b2ef-ed21b361f42d None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:21:20.400889 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-d94fbb20-7994-4ba4-b2ef-ed21b361f42d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:21:20.518939 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94fbb20-7994-4ba4-b2ef-ed21b361f42d None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:21:20.604348 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26dbf3a3-597a-4c7c-bc57-be9c407282ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3ec6ecb85f94ef59762877d9ea33ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:21:20.604989 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dbf3a3-597a-4c7c-bc57-be9c407282ae service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:20.605253 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dbf3a3-597a-4c7c-bc57-be9c407282ae service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:20.605518 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dbf3a3-597a-4c7c-bc57-be9c407282ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:20.607118 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-26dbf3a3-597a-4c7c-bc57-be9c407282ae service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:21:20.672757 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:21:20.731666 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26dbf3a3-597a-4c7c-bc57-be9c407282ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:21:20.733350 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26dbf3a3-597a-4c7c-bc57-be9c407282ae service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:21:20.754545 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 498/988] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:21:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:21:23.227855 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3702b795-62f4-4100-8f48-6f66b2f8f228 None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:23.228409 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3702b795-62f4-4100-8f48-6f66b2f8f228 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:23.228409 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3702b795-62f4-4100-8f48-6f66b2f8f228 None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:23.229772 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 491/989] 10.222.0.18 () {54 vars in 927 bytes} [Wed Nov 2 07:21:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:21:23.235301 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fb417b-bc48-430d-b65f-7e9a118295b0 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:23.235715 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fb417b-bc48-430d-b65f-7e9a118295b0 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:23.235823 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fb417b-bc48-430d-b65f-7e9a118295b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:23.316432 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-75fb417b-bc48-430d-b65f-7e9a118295b0 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:21:23.317616 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-75fb417b-bc48-430d-b65f-7e9a118295b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:21:23.413074 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 499/990] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Nov 2 07:21:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:21:23.692685 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3702b795-62f4-4100-8f48-6f66b2f8f228 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:21:23.694847 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-3702b795-62f4-4100-8f48-6f66b2f8f228 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:21:23.800329 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3702b795-62f4-4100-8f48-6f66b2f8f228 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:21:23.871159 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:21:23.873990 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7043aa6a-7187-45ff-8510-a268649ec030 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3ec6ecb85f94ef59762877d9ea33ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:21:23.874832 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7043aa6a-7187-45ff-8510-a268649ec030 service neutron] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:23.875300 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7043aa6a-7187-45ff-8510-a268649ec030 service neutron] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:23.875760 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7043aa6a-7187-45ff-8510-a268649ec030 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:23.877403 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-7043aa6a-7187-45ff-8510-a268649ec030 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:21:23.988051 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7043aa6a-7187-45ff-8510-a268649ec030 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:21:23.989380 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7043aa6a-7187-45ff-8510-a268649ec030 service neutron] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:21:24.008744 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 492/991] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:21:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:21:26.447647 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5bfa4b-a86a-4a9a-8a30-36ee52ff89cb None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:26.448226 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5bfa4b-a86a-4a9a-8a30-36ee52ff89cb None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:26.448226 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5bfa4b-a86a-4a9a-8a30-36ee52ff89cb None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:26.449519 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 500/992] 10.222.0.18 () {54 vars in 927 bytes} [Wed Nov 2 07:21:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:21:26.454885 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5df6a48-cb62-47bf-8f66-443d750ee99a None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:26.455307 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5df6a48-cb62-47bf-8f66-443d750ee99a None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:26.455427 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5df6a48-cb62-47bf-8f66-443d750ee99a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:26.521208 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-c5df6a48-cb62-47bf-8f66-443d750ee99a None None] MFA Rules not processed for user `7e7f81d50d1a49b3a0076a19ded69866`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:21:26.523168 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-c5df6a48-cb62-47bf-8f66-443d750ee99a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:21:26.574291 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:21:26.618234 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 493/993] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Nov 2 07:21:26 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:21:26.903388 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5bfa4b-a86a-4a9a-8a30-36ee52ff89cb None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:21:26.904918 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-6c5bfa4b-a86a-4a9a-8a30-36ee52ff89cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:21:26.914633 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:21:27.005318 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5bfa4b-a86a-4a9a-8a30-36ee52ff89cb None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:21:27.076471 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9dc65e4-7a80-4c46-90e7-5f6756e2c062 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3ec6ecb85f94ef59762877d9ea33ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:21:27.077056 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dc65e4-7a80-4c46-90e7-5f6756e2c062 service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:27.077332 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dc65e4-7a80-4c46-90e7-5f6756e2c062 service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:27.077598 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dc65e4-7a80-4c46-90e7-5f6756e2c062 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:27.079171 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-d9dc65e4-7a80-4c46-90e7-5f6756e2c062 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:21:27.141489 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:21:27.187683 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9dc65e4-7a80-4c46-90e7-5f6756e2c062 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:21:27.189045 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9dc65e4-7a80-4c46-90e7-5f6756e2c062 service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:21:27.206851 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 501/994] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:21:26 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:21:29.865436 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86526897-ba66-4c49-acae-fd3c25fbce89 None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:29.865990 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86526897-ba66-4c49-acae-fd3c25fbce89 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:29.865990 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86526897-ba66-4c49-acae-fd3c25fbce89 None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:29.867334 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 494/995] 10.222.0.18 () {54 vars in 927 bytes} [Wed Nov 2 07:21:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:21:29.872588 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb05ded-394b-4490-acf6-7b33c907041e None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:29.872959 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb05ded-394b-4490-acf6-7b33c907041e None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:29.873137 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb05ded-394b-4490-acf6-7b33c907041e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:29.941864 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-adb05ded-394b-4490-acf6-7b33c907041e None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:21:29.943022 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-adb05ded-394b-4490-acf6-7b33c907041e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:21:30.038573 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 502/996] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Nov 2 07:21:29 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:21:30.330083 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86526897-ba66-4c49-acae-fd3c25fbce89 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:21:30.331649 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-86526897-ba66-4c49-acae-fd3c25fbce89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:21:30.431657 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86526897-ba66-4c49-acae-fd3c25fbce89 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:21:30.503012 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c252eca-d58f-49ff-8c0a-2295ce9eda91 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3ec6ecb85f94ef59762877d9ea33ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:21:30.503605 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c252eca-d58f-49ff-8c0a-2295ce9eda91 service neutron] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:30.503885 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c252eca-d58f-49ff-8c0a-2295ce9eda91 service neutron] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:30.504152 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c252eca-d58f-49ff-8c0a-2295ce9eda91 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:30.505724 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-9c252eca-d58f-49ff-8c0a-2295ce9eda91 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:21:30.512963 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:21:30.613531 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c252eca-d58f-49ff-8c0a-2295ce9eda91 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:21:30.614921 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c252eca-d58f-49ff-8c0a-2295ce9eda91 service neutron] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:21:30.632889 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 495/997] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:21:30 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:21:33.404811 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9101255-0305-496e-b19a-4038ca45a656 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:33.405322 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9101255-0305-496e-b19a-4038ca45a656 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:33.405322 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9101255-0305-496e-b19a-4038ca45a656 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:33.406639 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 503/998] 10.222.0.18 () {54 vars in 927 bytes} [Wed Nov 2 07:21:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:21:33.411753 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fd84a9-ec34-4c1c-b0b1-47e577a0aa7d None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:33.412187 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fd84a9-ec34-4c1c-b0b1-47e577a0aa7d None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:33.412310 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fd84a9-ec34-4c1c-b0b1-47e577a0aa7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:33.487054 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-31fd84a9-ec34-4c1c-b0b1-47e577a0aa7d None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:21:33.488282 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-31fd84a9-ec34-4c1c-b0b1-47e577a0aa7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:21:33.550655 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:21:33.596385 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 496/999] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Nov 2 07:21:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:21:33.915510 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9061db1b-3e6b-4f03-a55a-23afe10148f7 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:33.915913 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9061db1b-3e6b-4f03-a55a-23afe10148f7 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:33.916256 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9061db1b-3e6b-4f03-a55a-23afe10148f7 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:33.917654 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 504/1000] 10.222.0.18 () {54 vars in 965 bytes} [Wed Nov 2 07:21:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Nov 02 07:21:33.923942 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2913248-54f3-4a35-98d2-66b7e7fe0dc8 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:33.924284 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2913248-54f3-4a35-98d2-66b7e7fe0dc8 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:33.924720 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2913248-54f3-4a35-98d2-66b7e7fe0dc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:33.999413 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-f2913248-54f3-4a35-98d2-66b7e7fe0dc8 None None] MFA Rules not processed for user `b3ec6ecb85f94ef59762877d9ea33ed0`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:21:34.000609 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-f2913248-54f3-4a35-98d2-66b7e7fe0dc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:21:34.076289 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 497/1001] 10.222.0.18 () {58 vars in 1062 bytes} [Wed Nov 2 07:21:33 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:21:34.085966 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9061db1b-3e6b-4f03-a55a-23afe10148f7 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:21:34.087487 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-9061db1b-3e6b-4f03-a55a-23afe10148f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:21:34.107058 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:21:34.189040 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9061db1b-3e6b-4f03-a55a-23afe10148f7 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:21:34.261033 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78daabf8-4e8e-46d6-b0a9-6153a3862c00 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3ec6ecb85f94ef59762877d9ea33ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:21:34.261630 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78daabf8-4e8e-46d6-b0a9-6153a3862c00 service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:34.261902 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78daabf8-4e8e-46d6-b0a9-6153a3862c00 service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:34.262163 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78daabf8-4e8e-46d6-b0a9-6153a3862c00 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:34.263739 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-78daabf8-4e8e-46d6-b0a9-6153a3862c00 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:21:34.333235 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:21:34.371822 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78daabf8-4e8e-46d6-b0a9-6153a3862c00 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:21:34.373238 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78daabf8-4e8e-46d6-b0a9-6153a3862c00 service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:21:34.390657 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 505/1002] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:21:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:21:36.860392 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc206c53-6673-4661-aeb1-5455d0a25a1a None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:36.861162 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc206c53-6673-4661-aeb1-5455d0a25a1a None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:36.861162 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc206c53-6673-4661-aeb1-5455d0a25a1a None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:36.862508 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 498/1003] 10.222.0.18 () {54 vars in 927 bytes} [Wed Nov 2 07:21:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:21:36.868432 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66928ac1-1108-495f-99ad-3e68844965e3 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:36.868837 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66928ac1-1108-495f-99ad-3e68844965e3 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:36.868977 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66928ac1-1108-495f-99ad-3e68844965e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:36.934061 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-66928ac1-1108-495f-99ad-3e68844965e3 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:21:36.935290 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-66928ac1-1108-495f-99ad-3e68844965e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:21:37.029140 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 506/1004] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Nov 2 07:21:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:21:37.311252 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc206c53-6673-4661-aeb1-5455d0a25a1a None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:21:37.312781 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-dc206c53-6673-4661-aeb1-5455d0a25a1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:21:37.374624 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:21:37.424516 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc206c53-6673-4661-aeb1-5455d0a25a1a None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:21:37.497762 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3da901a-0c52-4e2e-a6d0-6332f35ed476 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3ec6ecb85f94ef59762877d9ea33ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:21:37.498386 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3da901a-0c52-4e2e-a6d0-6332f35ed476 service neutron] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:37.498670 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3da901a-0c52-4e2e-a6d0-6332f35ed476 service neutron] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:37.498940 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3da901a-0c52-4e2e-a6d0-6332f35ed476 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:37.500516 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-d3da901a-0c52-4e2e-a6d0-6332f35ed476 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:21:37.613998 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:21:37.617093 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3da901a-0c52-4e2e-a6d0-6332f35ed476 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:21:37.618956 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3da901a-0c52-4e2e-a6d0-6332f35ed476 service neutron] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:21:37.637968 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 499/1005] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:21:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:21:39.868720 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ce6ca9-cb45-4ecb-b9b7-9b45450f6430 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:39.869262 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ce6ca9-cb45-4ecb-b9b7-9b45450f6430 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:39.869262 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ce6ca9-cb45-4ecb-b9b7-9b45450f6430 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:39.870635 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 507/1006] 10.222.0.18 () {54 vars in 927 bytes} [Wed Nov 2 07:21:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:21:39.876200 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fee0efd-7a2a-4a96-bdfa-8ba30def919e None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:39.876695 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fee0efd-7a2a-4a96-bdfa-8ba30def919e None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:39.876793 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fee0efd-7a2a-4a96-bdfa-8ba30def919e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:39.951657 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-9fee0efd-7a2a-4a96-bdfa-8ba30def919e None None] MFA Rules not processed for user `7e7f81d50d1a49b3a0076a19ded69866`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:21:39.952967 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-9fee0efd-7a2a-4a96-bdfa-8ba30def919e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:21:40.054684 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 500/1007] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Nov 2 07:21:39 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:21:40.341848 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ce6ca9-cb45-4ecb-b9b7-9b45450f6430 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:21:40.343515 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-00ce6ca9-cb45-4ecb-b9b7-9b45450f6430 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:21:40.376281 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:21:40.447528 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ce6ca9-cb45-4ecb-b9b7-9b45450f6430 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:21:40.523735 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ada35a9-a106-4326-82fe-dcb1184c3ef5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3ec6ecb85f94ef59762877d9ea33ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:21:40.524568 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ada35a9-a106-4326-82fe-dcb1184c3ef5 service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:40.524911 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ada35a9-a106-4326-82fe-dcb1184c3ef5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:40.525250 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ada35a9-a106-4326-82fe-dcb1184c3ef5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:40.526975 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-8ada35a9-a106-4326-82fe-dcb1184c3ef5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:21:40.608919 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:21:40.640478 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ada35a9-a106-4326-82fe-dcb1184c3ef5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:21:40.641987 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ada35a9-a106-4326-82fe-dcb1184c3ef5 service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:21:40.661414 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 508/1008] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:21:40 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:21:43.285794 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb772a3-ccc3-498e-8f6d-0653fa28ffee None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:43.286322 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb772a3-ccc3-498e-8f6d-0653fa28ffee None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:43.286637 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb772a3-ccc3-498e-8f6d-0653fa28ffee None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:43.288899 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 501/1009] 10.222.0.18 () {54 vars in 927 bytes} [Wed Nov 2 07:21:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:21:43.294503 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081d8c7f-aa35-4db7-b9be-7bb54daafac2 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:43.295207 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081d8c7f-aa35-4db7-b9be-7bb54daafac2 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:43.295207 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081d8c7f-aa35-4db7-b9be-7bb54daafac2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:43.373641 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-081d8c7f-aa35-4db7-b9be-7bb54daafac2 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:21:43.374802 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-081d8c7f-aa35-4db7-b9be-7bb54daafac2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:21:43.486511 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 509/1010] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Nov 2 07:21:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:21:43.775174 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb772a3-ccc3-498e-8f6d-0653fa28ffee None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:21:43.776728 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-7bb772a3-ccc3-498e-8f6d-0653fa28ffee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:21:43.845776 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:21:43.879912 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb772a3-ccc3-498e-8f6d-0653fa28ffee None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:21:43.954258 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95b8e23-4b5e-467f-a9cb-412ef20befd9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3ec6ecb85f94ef59762877d9ea33ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:21:43.954884 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95b8e23-4b5e-467f-a9cb-412ef20befd9 service neutron] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:43.955161 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95b8e23-4b5e-467f-a9cb-412ef20befd9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:43.955428 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95b8e23-4b5e-467f-a9cb-412ef20befd9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:43.957045 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-c95b8e23-4b5e-467f-a9cb-412ef20befd9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:21:44.070853 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95b8e23-4b5e-467f-a9cb-412ef20befd9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:21:44.072286 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95b8e23-4b5e-467f-a9cb-412ef20befd9 service neutron] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:21:44.081891 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:21:44.091649 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 502/1011] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:21:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:21:46.279548 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1ac800-2891-470c-b6ed-5a3743c9a91e None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:46.280115 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1ac800-2891-470c-b6ed-5a3743c9a91e None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:46.280115 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1ac800-2891-470c-b6ed-5a3743c9a91e None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:46.281443 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 510/1012] 10.222.0.18 () {54 vars in 927 bytes} [Wed Nov 2 07:21:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:21:46.286937 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37b5aa4-b95d-4397-b5f5-7bdf9664cd98 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:46.287908 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37b5aa4-b95d-4397-b5f5-7bdf9664cd98 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:46.288138 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37b5aa4-b95d-4397-b5f5-7bdf9664cd98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:46.352926 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-d37b5aa4-b95d-4397-b5f5-7bdf9664cd98 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:21:46.354117 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-d37b5aa4-b95d-4397-b5f5-7bdf9664cd98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:21:46.448603 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 503/1013] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Nov 2 07:21:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:21:46.733225 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1ac800-2891-470c-b6ed-5a3743c9a91e None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:21:46.734781 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-7b1ac800-2891-470c-b6ed-5a3743c9a91e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:21:46.774837 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:21:46.838133 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1ac800-2891-470c-b6ed-5a3743c9a91e None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:21:46.909728 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7613ac0-d8e9-4f3e-a578-1f8fa9a0bbf8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3ec6ecb85f94ef59762877d9ea33ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:21:46.910258 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7613ac0-d8e9-4f3e-a578-1f8fa9a0bbf8 service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:46.910568 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7613ac0-d8e9-4f3e-a578-1f8fa9a0bbf8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:46.910843 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7613ac0-d8e9-4f3e-a578-1f8fa9a0bbf8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:46.912419 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-e7613ac0-d8e9-4f3e-a578-1f8fa9a0bbf8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:21:46.995875 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:21:47.021255 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7613ac0-d8e9-4f3e-a578-1f8fa9a0bbf8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:21:47.022713 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7613ac0-d8e9-4f3e-a578-1f8fa9a0bbf8 service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:21:47.040517 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 511/1014] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:21:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:21:49.502868 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34420f1-5a80-4b31-ae0c-a1040cc9c3a7 None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:49.503411 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34420f1-5a80-4b31-ae0c-a1040cc9c3a7 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:49.503411 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34420f1-5a80-4b31-ae0c-a1040cc9c3a7 None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:49.504762 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 504/1015] 10.222.0.18 () {54 vars in 927 bytes} [Wed Nov 2 07:21:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:21:49.510387 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f124d43-af86-49f6-98f3-563a40655ec0 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:49.510823 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f124d43-af86-49f6-98f3-563a40655ec0 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:49.510953 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f124d43-af86-49f6-98f3-563a40655ec0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:49.575292 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-3f124d43-af86-49f6-98f3-563a40655ec0 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:21:49.576463 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-3f124d43-af86-49f6-98f3-563a40655ec0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:21:49.670145 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 512/1016] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Nov 2 07:21:49 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:21:49.954574 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34420f1-5a80-4b31-ae0c-a1040cc9c3a7 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:21:49.956139 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-f34420f1-5a80-4b31-ae0c-a1040cc9c3a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:21:50.035090 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:21:50.062600 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34420f1-5a80-4b31-ae0c-a1040cc9c3a7 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:21:50.135997 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a681b3-def6-40c4-8dde-2ba022fc54b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3ec6ecb85f94ef59762877d9ea33ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:21:50.136522 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a681b3-def6-40c4-8dde-2ba022fc54b0 service neutron] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:50.136801 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a681b3-def6-40c4-8dde-2ba022fc54b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:50.137070 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a681b3-def6-40c4-8dde-2ba022fc54b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:50.138679 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-d8a681b3-def6-40c4-8dde-2ba022fc54b0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:21:50.247016 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a681b3-def6-40c4-8dde-2ba022fc54b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:21:50.248438 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a681b3-def6-40c4-8dde-2ba022fc54b0 service neutron] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:21:50.265674 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 505/1017] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:21:49 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:21:54.455198 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53931c2-8074-48d1-abf2-f5549a4d1514 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:54.455706 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53931c2-8074-48d1-abf2-f5549a4d1514 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:54.455706 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53931c2-8074-48d1-abf2-f5549a4d1514 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:54.456965 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 513/1018] 10.222.0.18 () {54 vars in 927 bytes} [Wed Nov 2 07:21:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:21:54.462348 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4098a64b-a3e7-46c5-8bec-b1bf6c4c4a36 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:54.462804 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4098a64b-a3e7-46c5-8bec-b1bf6c4c4a36 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:54.462917 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4098a64b-a3e7-46c5-8bec-b1bf6c4c4a36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:54.468404 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:21:54.530463 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-4098a64b-a3e7-46c5-8bec-b1bf6c4c4a36 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:21:54.531676 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-4098a64b-a3e7-46c5-8bec-b1bf6c4c4a36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:21:54.628102 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 506/1019] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Nov 2 07:21:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:21:54.912863 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53931c2-8074-48d1-abf2-f5549a4d1514 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:21:54.915172 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-f53931c2-8074-48d1-abf2-f5549a4d1514 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:21:54.964042 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:21:55.019047 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53931c2-8074-48d1-abf2-f5549a4d1514 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:21:55.091703 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330ed991-a3d8-40f1-aba3-cb0765f59e65 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3ec6ecb85f94ef59762877d9ea33ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:21:55.092307 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330ed991-a3d8-40f1-aba3-cb0765f59e65 service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:55.092582 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330ed991-a3d8-40f1-aba3-cb0765f59e65 service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:55.092852 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330ed991-a3d8-40f1-aba3-cb0765f59e65 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:55.094444 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-330ed991-a3d8-40f1-aba3-cb0765f59e65 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:21:55.187319 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:21:55.203493 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330ed991-a3d8-40f1-aba3-cb0765f59e65 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:21:55.204915 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330ed991-a3d8-40f1-aba3-cb0765f59e65 service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:21:55.222531 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 514/1020] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:21:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:21:58.022680 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c44da96-4022-4826-883c-9fd723ebf6f5 None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:58.023248 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c44da96-4022-4826-883c-9fd723ebf6f5 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:58.023248 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c44da96-4022-4826-883c-9fd723ebf6f5 None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:58.024743 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 507/1021] 10.222.0.18 () {54 vars in 927 bytes} [Wed Nov 2 07:21:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:21:58.030300 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d9e06e-e14b-4b78-b9b5-0ba18597b868 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:58.030745 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d9e06e-e14b-4b78-b9b5-0ba18597b868 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:58.030878 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d9e06e-e14b-4b78-b9b5-0ba18597b868 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:58.099303 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-d7d9e06e-e14b-4b78-b9b5-0ba18597b868 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:21:58.100490 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-d7d9e06e-e14b-4b78-b9b5-0ba18597b868 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:21:58.198244 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 515/1022] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Nov 2 07:21:58 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:21:58.485644 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c44da96-4022-4826-883c-9fd723ebf6f5 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:21:58.487576 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-9c44da96-4022-4826-883c-9fd723ebf6f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:21:58.575508 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:21:58.594078 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c44da96-4022-4826-883c-9fd723ebf6f5 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:21:58.673291 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8f0468-47a9-40f3-90b2-de9e3de89b2e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3ec6ecb85f94ef59762877d9ea33ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:21:58.673888 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8f0468-47a9-40f3-90b2-de9e3de89b2e service neutron] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:58.674164 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8f0468-47a9-40f3-90b2-de9e3de89b2e service neutron] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:58.674460 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8f0468-47a9-40f3-90b2-de9e3de89b2e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:58.676147 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-8a8f0468-47a9-40f3-90b2-de9e3de89b2e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:21:58.786575 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8f0468-47a9-40f3-90b2-de9e3de89b2e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:21:58.788011 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8f0468-47a9-40f3-90b2-de9e3de89b2e service neutron] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:21:58.808981 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 508/1023] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:21:58 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:22:03.311428 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45adae8-5d4a-4f20-b68d-9ad411237a1c None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:22:03.311986 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45adae8-5d4a-4f20-b68d-9ad411237a1c None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:22:03.311986 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45adae8-5d4a-4f20-b68d-9ad411237a1c None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:22:03.313307 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 516/1024] 10.222.0.18 () {54 vars in 927 bytes} [Wed Nov 2 07:22:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:22:03.318836 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325007ae-610a-4282-9a3e-33b0c02d8e46 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:22:03.319254 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325007ae-610a-4282-9a3e-33b0c02d8e46 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:22:03.319396 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325007ae-610a-4282-9a3e-33b0c02d8e46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:22:03.338136 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:22:03.396066 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-325007ae-610a-4282-9a3e-33b0c02d8e46 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:22:03.397368 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-325007ae-610a-4282-9a3e-33b0c02d8e46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:22:03.501548 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 509/1025] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Nov 2 07:22:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:22:03.797747 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45adae8-5d4a-4f20-b68d-9ad411237a1c None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:22:03.799368 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-a45adae8-5d4a-4f20-b68d-9ad411237a1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:22:03.857190 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:22:03.903046 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45adae8-5d4a-4f20-b68d-9ad411237a1c None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:22:03.975221 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7467714c-cae3-4fda-b9b4-dcc7fdd73fa9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3ec6ecb85f94ef59762877d9ea33ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:22:03.975822 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7467714c-cae3-4fda-b9b4-dcc7fdd73fa9 service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:22:03.976097 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7467714c-cae3-4fda-b9b4-dcc7fdd73fa9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:22:03.976364 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7467714c-cae3-4fda-b9b4-dcc7fdd73fa9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:22:03.978963 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-7467714c-cae3-4fda-b9b4-dcc7fdd73fa9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:22:04.083950 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:22:04.088527 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7467714c-cae3-4fda-b9b4-dcc7fdd73fa9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:22:04.089946 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7467714c-cae3-4fda-b9b4-dcc7fdd73fa9 service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:22:04.108040 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 517/1026] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:22:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:22:09.040451 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f4edb6-7552-4c0b-ab31-9ddad6276788 None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:22:09.042044 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f4edb6-7552-4c0b-ab31-9ddad6276788 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:22:09.044014 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f4edb6-7552-4c0b-ab31-9ddad6276788 None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:22:09.050322 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 510/1027] 10.222.0.18 () {54 vars in 921 bytes} [Wed Nov 2 07:22:09 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:22:09.058986 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf2d25e-dd79-42ee-acb5-987fb4e31f37 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:22:09.059837 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf2d25e-dd79-42ee-acb5-987fb4e31f37 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:22:09.060448 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf2d25e-dd79-42ee-acb5-987fb4e31f37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:22:09.189766 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-8cf2d25e-dd79-42ee-acb5-987fb4e31f37 None None] MFA Rules not processed for user `4f2fc6a156c54d0c9d325901d0f7f6fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:22:09.191710 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-8cf2d25e-dd79-42ee-acb5-987fb4e31f37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:22:09.315406 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 518/1028] 10.222.0.18 () {58 vars in 1018 bytes} [Wed Nov 2 07:22:09 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:22:13.353286 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d6740d-dd73-4b30-ab30-70d771d0754d None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:22:13.354219 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d6740d-dd73-4b30-ab30-70d771d0754d None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:22:13.354634 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d6740d-dd73-4b30-ab30-70d771d0754d None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:22:13.356093 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 511/1029] 10.222.0.18 () {54 vars in 921 bytes} [Wed Nov 2 07:22:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:22:13.361693 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270d3f91-33d7-47e2-a112-3daa09663ab5 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:22:13.362148 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270d3f91-33d7-47e2-a112-3daa09663ab5 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:22:13.362230 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270d3f91-33d7-47e2-a112-3daa09663ab5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:22:13.418741 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:22:13.444760 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-270d3f91-33d7-47e2-a112-3daa09663ab5 None None] MFA Rules not processed for user `4f2fc6a156c54d0c9d325901d0f7f6fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:22:13.446065 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-270d3f91-33d7-47e2-a112-3daa09663ab5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:22:13.499621 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=59 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:22:13.561175 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 519/1030] 10.222.0.18 () {58 vars in 1018 bytes} [Wed Nov 2 07:22:13 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:22:15.028206 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22262977-02c7-4b13-bebd-43043a798dbf None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:22:15.028756 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22262977-02c7-4b13-bebd-43043a798dbf None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:22:15.028875 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22262977-02c7-4b13-bebd-43043a798dbf None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:22:15.030224 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 512/1031] 10.222.0.18 () {54 vars in 921 bytes} [Wed Nov 2 07:22:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:22:15.035417 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147092f6-ab3f-4826-b0d0-e8a5274555f0 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:22:15.035829 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147092f6-ab3f-4826-b0d0-e8a5274555f0 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:22:15.035967 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147092f6-ab3f-4826-b0d0-e8a5274555f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:22:15.114675 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-147092f6-ab3f-4826-b0d0-e8a5274555f0 None None] MFA Rules not processed for user `4f2fc6a156c54d0c9d325901d0f7f6fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:22:15.115883 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-147092f6-ab3f-4826-b0d0-e8a5274555f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:22:15.161687 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:22:15.229044 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 520/1032] 10.222.0.18 () {58 vars in 1018 bytes} [Wed Nov 2 07:22:15 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:22:18.856285 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac1e312-0e44-4136-8b80-c39771262664 None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:22:18.856813 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac1e312-0e44-4136-8b80-c39771262664 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:22:18.856813 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac1e312-0e44-4136-8b80-c39771262664 None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:22:18.858159 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 513/1033] 10.222.0.18 () {54 vars in 927 bytes} [Wed Nov 2 07:22:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:22:18.863860 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88e5c5b-2319-4ea6-9c9c-448953b100cb None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:22:18.864297 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88e5c5b-2319-4ea6-9c9c-448953b100cb None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:22:18.864400 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88e5c5b-2319-4ea6-9c9c-448953b100cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:22:18.932221 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-d88e5c5b-2319-4ea6-9c9c-448953b100cb None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:22:18.933399 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-d88e5c5b-2319-4ea6-9c9c-448953b100cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:22:19.035045 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 521/1034] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Nov 2 07:22:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:22:19.338148 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290c6264-7251-4a37-869c-657417caaa57 None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:22:19.338947 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290c6264-7251-4a37-869c-657417caaa57 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:22:19.340413 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290c6264-7251-4a37-869c-657417caaa57 None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:22:19.344338 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 514/1035] 10.222.0.18 () {54 vars in 961 bytes} [Wed Nov 2 07:22:19 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:22:19.353216 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0d9b19-68d3-4a82-8531-44af9798f6fd None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:22:19.353580 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0d9b19-68d3-4a82-8531-44af9798f6fd None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:22:19.353832 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0d9b19-68d3-4a82-8531-44af9798f6fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:22:19.375934 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:22:19.443305 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-8a0d9b19-68d3-4a82-8531-44af9798f6fd None None] MFA Rules not processed for user `69e48cba3f804cf08b2477f01ae95179`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:22:19.444443 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-8a0d9b19-68d3-4a82-8531-44af9798f6fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:22:19.566967 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 522/1036] 10.222.0.18 () {58 vars in 1058 bytes} [Wed Nov 2 07:22:19 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:22:19.575174 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290c6264-7251-4a37-869c-657417caaa57 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:22:19.576672 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-290c6264-7251-4a37-869c-657417caaa57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:22:19.736880 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290c6264-7251-4a37-869c-657417caaa57 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:22:19.812547 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:22:19.862226 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c6d78e-82a5-4e98-8007-d13511154766 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e48cba3f804cf08b2477f01ae95179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:22:19.863112 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c6d78e-82a5-4e98-8007-d13511154766 service nova] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:22:19.863384 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c6d78e-82a5-4e98-8007-d13511154766 service nova] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:22:19.863654 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c6d78e-82a5-4e98-8007-d13511154766 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:22:19.865238 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-24c6d78e-82a5-4e98-8007-d13511154766 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:22:19.968260 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c6d78e-82a5-4e98-8007-d13511154766 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:22:19.969921 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c6d78e-82a5-4e98-8007-d13511154766 service nova] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:22:19.989748 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 515/1037] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:22:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:22:22.387969 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e7ceea-6acb-4e19-9287-b7e5a876bb87 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:22:22.388507 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e7ceea-6acb-4e19-9287-b7e5a876bb87 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:22:22.388646 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e7ceea-6acb-4e19-9287-b7e5a876bb87 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:22:22.389866 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 523/1038] 10.222.0.18 () {54 vars in 927 bytes} [Wed Nov 2 07:22:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:22:22.395281 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3637994b-9ca7-421b-89c3-449feaffdc6a None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:22:22.395745 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3637994b-9ca7-421b-89c3-449feaffdc6a None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:22:22.395828 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3637994b-9ca7-421b-89c3-449feaffdc6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:22:22.464347 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-3637994b-9ca7-421b-89c3-449feaffdc6a None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:22:22.465521 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-3637994b-9ca7-421b-89c3-449feaffdc6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:22:22.504006 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:22:22.568149 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 516/1039] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Nov 2 07:22:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:22:22.856572 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e7ceea-6acb-4e19-9287-b7e5a876bb87 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:22:22.858081 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-84e7ceea-6acb-4e19-9287-b7e5a876bb87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:22:22.933644 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:22:23.036017 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e7ceea-6acb-4e19-9287-b7e5a876bb87 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:22:23.176288 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089a0855-341a-4b62-9992-133e3acfbdb4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e48cba3f804cf08b2477f01ae95179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:22:23.176288 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089a0855-341a-4b62-9992-133e3acfbdb4 service nova] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:22:23.176833 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089a0855-341a-4b62-9992-133e3acfbdb4 service nova] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:22:23.176833 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089a0855-341a-4b62-9992-133e3acfbdb4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:22:23.178614 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-089a0855-341a-4b62-9992-133e3acfbdb4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:22:23.202610 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:22:23.273364 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089a0855-341a-4b62-9992-133e3acfbdb4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:22:23.275695 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089a0855-341a-4b62-9992-133e3acfbdb4 service nova] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:22:23.294799 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 524/1040] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:22:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:22:25.780738 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfce556-e931-421e-a261-69a55ffd6b62 None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:22:25.781297 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfce556-e931-421e-a261-69a55ffd6b62 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:22:25.781297 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfce556-e931-421e-a261-69a55ffd6b62 None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:22:25.782728 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 517/1041] 10.222.0.18 () {54 vars in 927 bytes} [Wed Nov 2 07:22:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:22:25.788801 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a270ae1d-1190-4b5c-9625-7d2b51745146 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:22:25.790208 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a270ae1d-1190-4b5c-9625-7d2b51745146 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:22:25.790545 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a270ae1d-1190-4b5c-9625-7d2b51745146 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:22:25.857261 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-a270ae1d-1190-4b5c-9625-7d2b51745146 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:22:25.858487 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-a270ae1d-1190-4b5c-9625-7d2b51745146 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:22:25.953616 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:22:25.966793 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 525/1042] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Nov 2 07:22:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:22:26.244618 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c2f20e-0d20-4fe8-a8aa-774787c5edfc None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:22:26.244978 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c2f20e-0d20-4fe8-a8aa-774787c5edfc None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:22:26.245166 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c2f20e-0d20-4fe8-a8aa-774787c5edfc None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:22:26.246875 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 518/1043] 10.222.0.18 () {54 vars in 961 bytes} [Wed Nov 2 07:22:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:22:26.252727 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b07d9f-34b0-470e-8604-7216181862f1 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:22:26.253041 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b07d9f-34b0-470e-8604-7216181862f1 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:22:26.253300 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b07d9f-34b0-470e-8604-7216181862f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:22:26.337170 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-11b07d9f-34b0-470e-8604-7216181862f1 None None] MFA Rules not processed for user `69e48cba3f804cf08b2477f01ae95179`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:22:26.338468 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-11b07d9f-34b0-470e-8604-7216181862f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:22:26.458413 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 526/1044] 10.222.0.18 () {58 vars in 1058 bytes} [Wed Nov 2 07:22:26 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:22:26.464970 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c2f20e-0d20-4fe8-a8aa-774787c5edfc None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:22:26.466474 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-b4c2f20e-0d20-4fe8-a8aa-774787c5edfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:22:26.616146 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c2f20e-0d20-4fe8-a8aa-774787c5edfc None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:22:26.637471 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:22:26.746564 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3187fe74-dbd5-4fc7-9240-b72791147451 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e48cba3f804cf08b2477f01ae95179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:22:26.747178 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3187fe74-dbd5-4fc7-9240-b72791147451 service nova] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:22:26.747455 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3187fe74-dbd5-4fc7-9240-b72791147451 service nova] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:22:26.747723 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3187fe74-dbd5-4fc7-9240-b72791147451 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:22:26.749488 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-3187fe74-dbd5-4fc7-9240-b72791147451 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:22:26.844929 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3187fe74-dbd5-4fc7-9240-b72791147451 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:22:26.846672 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3187fe74-dbd5-4fc7-9240-b72791147451 service nova] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:22:26.865726 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 519/1045] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:22:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:22:27.081573 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc995521-b8e2-438a-89e2-f6c65094d619 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:22:27.082152 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc995521-b8e2-438a-89e2-f6c65094d619 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:22:27.082152 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc995521-b8e2-438a-89e2-f6c65094d619 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:22:27.083679 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 527/1046] 10.222.0.18 () {54 vars in 927 bytes} [Wed Nov 2 07:22:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:22:27.090012 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1656a1-0857-43b7-819c-245019d61e4d None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:22:27.090307 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1656a1-0857-43b7-819c-245019d61e4d None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:22:27.090603 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1656a1-0857-43b7-819c-245019d61e4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:22:27.129124 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:22:27.171529 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-5c1656a1-0857-43b7-819c-245019d61e4d None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:22:27.172870 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-5c1656a1-0857-43b7-819c-245019d61e4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:22:27.285072 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 520/1047] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Nov 2 07:22:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:22:27.626632 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cf8499-1e6c-4b83-a88d-79c40b124141 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:22:27.626632 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cf8499-1e6c-4b83-a88d-79c40b124141 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:22:27.627235 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cf8499-1e6c-4b83-a88d-79c40b124141 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:22:27.628561 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 528/1048] 10.222.0.18 () {54 vars in 963 bytes} [Wed Nov 2 07:22:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:22:27.633559 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf181e7d-95d6-4492-bf06-d9333bac25fe None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:22:27.634005 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf181e7d-95d6-4492-bf06-d9333bac25fe None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:22:27.634247 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf181e7d-95d6-4492-bf06-d9333bac25fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:22:27.715465 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-bf181e7d-95d6-4492-bf06-d9333bac25fe None None] MFA Rules not processed for user `43bb326fb82e4d2c83fb2eaf91860dc8`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:22:27.716698 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-bf181e7d-95d6-4492-bf06-d9333bac25fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:22:27.766694 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:22:27.799913 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 521/1049] 10.222.0.18 () {58 vars in 1060 bytes} [Wed Nov 2 07:22:27 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:22:27.807756 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cf8499-1e6c-4b83-a88d-79c40b124141 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:22:27.809379 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-88cf8499-1e6c-4b83-a88d-79c40b124141 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:22:27.870864 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:22:27.928175 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cf8499-1e6c-4b83-a88d-79c40b124141 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:22:28.014664 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad0d768-0de1-4153-aa36-54201ad0c60c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43bb326fb82e4d2c83fb2eaf91860dc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:22:28.015344 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad0d768-0de1-4153-aa36-54201ad0c60c service cinder] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:22:28.015696 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad0d768-0de1-4153-aa36-54201ad0c60c service cinder] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:22:28.016029 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad0d768-0de1-4153-aa36-54201ad0c60c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:22:28.017939 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-1ad0d768-0de1-4153-aa36-54201ad0c60c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:22:28.132948 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:22:28.155095 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad0d768-0de1-4153-aa36-54201ad0c60c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:22:28.156533 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad0d768-0de1-4153-aa36-54201ad0c60c service cinder] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:22:28.176862 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 529/1050] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:22:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:22:29.105825 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db2f1f8-4359-40be-80b6-4b1f5ca044c6 None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:22:29.106402 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db2f1f8-4359-40be-80b6-4b1f5ca044c6 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:22:29.106402 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db2f1f8-4359-40be-80b6-4b1f5ca044c6 None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:22:29.107807 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 522/1051] 10.222.0.18 () {54 vars in 927 bytes} [Wed Nov 2 07:22:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:22:29.113078 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7d2586-075a-408a-9b4e-d15bba74e354 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:22:29.113626 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7d2586-075a-408a-9b4e-d15bba74e354 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:22:29.113900 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7d2586-075a-408a-9b4e-d15bba74e354 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:22:29.181284 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-8e7d2586-075a-408a-9b4e-d15bba74e354 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:22:29.182496 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-8e7d2586-075a-408a-9b4e-d15bba74e354 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:22:29.277415 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 530/1052] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Nov 2 07:22:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:22:29.567302 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db2f1f8-4359-40be-80b6-4b1f5ca044c6 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:22:29.568934 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-2db2f1f8-4359-40be-80b6-4b1f5ca044c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:22:29.737679 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db2f1f8-4359-40be-80b6-4b1f5ca044c6 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:22:29.797955 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:22:29.863118 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d550215-0a9b-4eed-9de5-0aa40e22fd80 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e48cba3f804cf08b2477f01ae95179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:22:29.863986 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d550215-0a9b-4eed-9de5-0aa40e22fd80 service nova] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:22:29.864260 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d550215-0a9b-4eed-9de5-0aa40e22fd80 service nova] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:22:29.864525 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d550215-0a9b-4eed-9de5-0aa40e22fd80 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:22:29.866096 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-9d550215-0a9b-4eed-9de5-0aa40e22fd80 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:22:29.957730 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d550215-0a9b-4eed-9de5-0aa40e22fd80 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:22:29.959470 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d550215-0a9b-4eed-9de5-0aa40e22fd80 service nova] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:22:29.978231 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 523/1053] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:22:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:22:32.214006 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d14fe8-c02b-4ef5-bad5-cf36dabd75db None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:22:32.214669 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d14fe8-c02b-4ef5-bad5-cf36dabd75db None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:22:32.214669 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d14fe8-c02b-4ef5-bad5-cf36dabd75db None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:22:32.215882 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 531/1054] 10.222.0.18 () {54 vars in 927 bytes} [Wed Nov 2 07:22:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:22:32.221172 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fc5807-27f6-4c8e-9b17-fa3dc1a73f9f None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:22:32.221591 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fc5807-27f6-4c8e-9b17-fa3dc1a73f9f None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:22:32.221716 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fc5807-27f6-4c8e-9b17-fa3dc1a73f9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:22:32.290043 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-f5fc5807-27f6-4c8e-9b17-fa3dc1a73f9f None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:22:32.291895 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-f5fc5807-27f6-4c8e-9b17-fa3dc1a73f9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:22:32.299457 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:22:32.402164 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 524/1055] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Nov 2 07:22:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:22:32.703189 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d14fe8-c02b-4ef5-bad5-cf36dabd75db None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:22:32.705112 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-f6d14fe8-c02b-4ef5-bad5-cf36dabd75db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:22:32.770431 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:22:32.861261 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d14fe8-c02b-4ef5-bad5-cf36dabd75db None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:22:32.981263 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4b6d0a-b820-4919-be9f-22ff57a63326 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e48cba3f804cf08b2477f01ae95179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:22:32.981783 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4b6d0a-b820-4919-be9f-22ff57a63326 service nova] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:22:32.982062 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4b6d0a-b820-4919-be9f-22ff57a63326 service nova] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:22:32.982331 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4b6d0a-b820-4919-be9f-22ff57a63326 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:22:32.983908 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-5d4b6d0a-b820-4919-be9f-22ff57a63326 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:22:33.004153 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:22:33.082049 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4b6d0a-b820-4919-be9f-22ff57a63326 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:22:33.083671 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4b6d0a-b820-4919-be9f-22ff57a63326 service nova] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:22:33.102206 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 532/1056] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:22:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:22:35.365657 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658ac06a-aa4d-4ca5-853a-ef856b12366b None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:22:35.366233 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658ac06a-aa4d-4ca5-853a-ef856b12366b None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:22:35.366233 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658ac06a-aa4d-4ca5-853a-ef856b12366b None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:22:35.367589 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 525/1057] 10.222.0.18 () {54 vars in 927 bytes} [Wed Nov 2 07:22:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:22:35.373660 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec69dbb2-6b9a-47a4-a824-a821b233687e None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:22:35.374099 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec69dbb2-6b9a-47a4-a824-a821b233687e None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:22:35.374232 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec69dbb2-6b9a-47a4-a824-a821b233687e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:22:35.447826 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-ec69dbb2-6b9a-47a4-a824-a821b233687e None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:22:35.449002 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-ec69dbb2-6b9a-47a4-a824-a821b233687e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:22:35.562570 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 533/1058] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Nov 2 07:22:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:22:35.566900 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:22:35.859118 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658ac06a-aa4d-4ca5-853a-ef856b12366b None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:22:35.860571 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-658ac06a-aa4d-4ca5-853a-ef856b12366b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:22:36.028730 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:22:36.040446 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658ac06a-aa4d-4ca5-853a-ef856b12366b None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:22:36.170922 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7c02d4-bce6-4f45-b559-9c6bd169483c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e48cba3f804cf08b2477f01ae95179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:22:36.171449 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7c02d4-bce6-4f45-b559-9c6bd169483c service nova] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:22:36.171725 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7c02d4-bce6-4f45-b559-9c6bd169483c service nova] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:22:36.172001 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7c02d4-bce6-4f45-b559-9c6bd169483c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:22:36.173687 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-dd7c02d4-bce6-4f45-b559-9c6bd169483c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:22:36.274930 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7c02d4-bce6-4f45-b559-9c6bd169483c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:22:36.276533 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7c02d4-bce6-4f45-b559-9c6bd169483c service nova] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:22:36.288165 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:22:36.298385 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 526/1059] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:22:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:22:38.816446 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5980d829-4a9b-4884-a045-3f6ec0d9c058 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:22:38.817009 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5980d829-4a9b-4884-a045-3f6ec0d9c058 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:22:38.817192 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5980d829-4a9b-4884-a045-3f6ec0d9c058 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:22:38.819046 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 534/1060] 10.222.0.18 () {54 vars in 927 bytes} [Wed Nov 2 07:22:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:22:38.825130 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ab48c0-ce18-4962-9764-a9a2deb04b07 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:22:38.825729 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ab48c0-ce18-4962-9764-a9a2deb04b07 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:22:38.825729 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ab48c0-ce18-4962-9764-a9a2deb04b07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:22:38.909218 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-51ab48c0-ce18-4962-9764-a9a2deb04b07 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:22:38.910858 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-51ab48c0-ce18-4962-9764-a9a2deb04b07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:22:39.016394 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 527/1061] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Nov 2 07:22:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:22:39.316155 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5980d829-4a9b-4884-a045-3f6ec0d9c058 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:22:39.317687 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-5980d829-4a9b-4884-a045-3f6ec0d9c058 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:22:39.471931 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5980d829-4a9b-4884-a045-3f6ec0d9c058 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:22:39.543768 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:22:39.592325 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ba01a5-0f3d-452f-8d1b-62f45b994e68 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e48cba3f804cf08b2477f01ae95179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:22:39.593117 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ba01a5-0f3d-452f-8d1b-62f45b994e68 service nova] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:22:39.593476 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ba01a5-0f3d-452f-8d1b-62f45b994e68 service nova] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:22:39.593829 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ba01a5-0f3d-452f-8d1b-62f45b994e68 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:22:39.596074 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-31ba01a5-0f3d-452f-8d1b-62f45b994e68 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:22:39.711026 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ba01a5-0f3d-452f-8d1b-62f45b994e68 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:22:39.713245 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ba01a5-0f3d-452f-8d1b-62f45b994e68 service nova] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:22:39.732154 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 535/1062] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:22:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:22:42.065146 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a71d763-d131-4a82-bf33-b66a8f3c57d8 None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:22:42.065837 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a71d763-d131-4a82-bf33-b66a8f3c57d8 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:22:42.065837 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a71d763-d131-4a82-bf33-b66a8f3c57d8 None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:22:42.067079 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 528/1063] 10.222.0.18 () {54 vars in 927 bytes} [Wed Nov 2 07:22:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:22:42.072433 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2b368b-1223-4e80-9ca5-aadb58892b6e None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:22:42.072879 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2b368b-1223-4e80-9ca5-aadb58892b6e None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:22:42.073078 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2b368b-1223-4e80-9ca5-aadb58892b6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:22:42.141074 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-8f2b368b-1223-4e80-9ca5-aadb58892b6e None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:22:42.142267 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-8f2b368b-1223-4e80-9ca5-aadb58892b6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:22:42.176739 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:22:42.241202 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 536/1064] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Nov 2 07:22:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:22:42.529466 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a71d763-d131-4a82-bf33-b66a8f3c57d8 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:22:42.531030 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-1a71d763-d131-4a82-bf33-b66a8f3c57d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:22:42.607013 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:22:42.685864 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a71d763-d131-4a82-bf33-b66a8f3c57d8 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:22:42.806326 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2ff3e1-72cf-4b8a-a0a1-ed4cee592ee3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e48cba3f804cf08b2477f01ae95179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:22:42.807291 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2ff3e1-72cf-4b8a-a0a1-ed4cee592ee3 service nova] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:22:42.807625 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2ff3e1-72cf-4b8a-a0a1-ed4cee592ee3 service nova] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:22:42.807896 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2ff3e1-72cf-4b8a-a0a1-ed4cee592ee3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:22:42.809503 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-ab2ff3e1-72cf-4b8a-a0a1-ed4cee592ee3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:22:42.849217 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:22:42.902760 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2ff3e1-72cf-4b8a-a0a1-ed4cee592ee3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:22:42.904379 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2ff3e1-72cf-4b8a-a0a1-ed4cee592ee3 service nova] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:22:42.923011 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 529/1065] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:22:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:22:43.775691 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e9bd3f-dc07-4bca-a41e-23cc03fff90e None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:22:43.776253 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e9bd3f-dc07-4bca-a41e-23cc03fff90e None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:22:43.776253 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e9bd3f-dc07-4bca-a41e-23cc03fff90e None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:22:43.777592 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 537/1066] 10.222.0.18 () {54 vars in 927 bytes} [Wed Nov 2 07:22:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:22:43.782893 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ac670f-a4c8-4bda-b098-e710c106ee6f None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:22:43.783423 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ac670f-a4c8-4bda-b098-e710c106ee6f None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:22:43.783423 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ac670f-a4c8-4bda-b098-e710c106ee6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:22:43.850085 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-a8ac670f-a4c8-4bda-b098-e710c106ee6f None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:22:43.851335 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-a8ac670f-a4c8-4bda-b098-e710c106ee6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:22:43.953449 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 530/1067] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Nov 2 07:22:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:22:44.269051 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2372484-0472-4cde-bc22-5bbcb3d5cb4d None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:22:44.269417 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2372484-0472-4cde-bc22-5bbcb3d5cb4d None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:22:44.269782 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2372484-0472-4cde-bc22-5bbcb3d5cb4d None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:22:44.271243 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 538/1068] 10.222.0.18 () {54 vars in 962 bytes} [Wed Nov 2 07:22:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:22:44.276820 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a55839-3785-4864-9197-3e7a8f2bd42b None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:22:44.277114 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a55839-3785-4864-9197-3e7a8f2bd42b None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:22:44.277393 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a55839-3785-4864-9197-3e7a8f2bd42b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:22:44.294162 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:22:44.356036 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-d4a55839-3785-4864-9197-3e7a8f2bd42b None None] MFA Rules not processed for user `c5353f2af0f44d668e2a269647aa1dc2`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:22:44.357220 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-d4a55839-3785-4864-9197-3e7a8f2bd42b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:22:44.435681 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 531/1069] 10.222.0.18 () {58 vars in 1059 bytes} [Wed Nov 2 07:22:44 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:22:44.443407 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2372484-0472-4cde-bc22-5bbcb3d5cb4d None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:22:44.444910 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-b2372484-0472-4cde-bc22-5bbcb3d5cb4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:22:44.547009 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2372484-0472-4cde-bc22-5bbcb3d5cb4d None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:22:44.608397 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:22:44.624185 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac26445-613a-4a72-ba67-44b019a00227 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5353f2af0f44d668e2a269647aa1dc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:22:44.624821 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac26445-613a-4a72-ba67-44b019a00227 service glance] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:22:44.625099 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac26445-613a-4a72-ba67-44b019a00227 service glance] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:22:44.625365 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac26445-613a-4a72-ba67-44b019a00227 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:22:44.626960 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-2ac26445-613a-4a72-ba67-44b019a00227 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:22:44.744588 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac26445-613a-4a72-ba67-44b019a00227 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:22:44.746022 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac26445-613a-4a72-ba67-44b019a00227 service glance] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:22:44.763612 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 539/1070] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:22:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 07:22:45.166190 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac52c4d-87e7-4e54-9e8e-36f91f453f28 None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:22:45.166784 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac52c4d-87e7-4e54-9e8e-36f91f453f28 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:22:45.166784 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac52c4d-87e7-4e54-9e8e-36f91f453f28 None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:22:45.168158 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 532/1071] 10.222.0.18 () {54 vars in 927 bytes} [Wed Nov 2 07:22:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:22:45.173378 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b36205-faa7-4553-9c11-cf8b9943f007 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:22:45.173776 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b36205-faa7-4553-9c11-cf8b9943f007 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:22:45.173930 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b36205-faa7-4553-9c11-cf8b9943f007 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:22:45.240223 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-23b36205-faa7-4553-9c11-cf8b9943f007 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:22:45.241412 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-23b36205-faa7-4553-9c11-cf8b9943f007 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:22:45.289019 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:22:45.355969 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 540/1072] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Nov 2 07:22:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:22:45.534251 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcb6be3-ef25-4106-be01-32950af6d7e1 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:22:45.534551 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcb6be3-ef25-4106-be01-32950af6d7e1 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:22:45.534823 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcb6be3-ef25-4106-be01-32950af6d7e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:22:45.607263 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-7bcb6be3-ef25-4106-be01-32950af6d7e1 None None] MFA Rules not processed for user `ff4736f3d50746b0bf5d6a917b2ca0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:22:45.608445 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-7bcb6be3-ef25-4106-be01-32950af6d7e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:22:45.662783 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b36205-faa7-4553-9c11-cf8b9943f007 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:22:45.664379 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-23b36205-faa7-4553-9c11-cf8b9943f007 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:22:45.670000 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:22:45.702592 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 533/1073] 10.222.0.18 () {58 vars in 1009 bytes} [Wed Nov 2 07:22:45 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:22:45.747894 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3003d1f8-0a4d-4255-90b0-4d3bd9d376d0 None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:22:45.748168 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3003d1f8-0a4d-4255-90b0-4d3bd9d376d0 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:22:45.748442 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3003d1f8-0a4d-4255-90b0-4d3bd9d376d0 None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:22:45.749790 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 534/1074] 10.222.0.18 () {54 vars in 943 bytes} [Wed Nov 2 07:22:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Nov 02 07:22:45.755948 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fbb714-953d-4ad8-90fb-6abb50788bd3 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:22:45.756231 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fbb714-953d-4ad8-90fb-6abb50788bd3 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:22:45.756502 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fbb714-953d-4ad8-90fb-6abb50788bd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:22:45.783463 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b36205-faa7-4553-9c11-cf8b9943f007 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:22:45.835127 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-a9fbb714-953d-4ad8-90fb-6abb50788bd3 None None] MFA Rules not processed for user `f9116f47889f4db39e0203d6cdae1593`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:22:45.836326 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-a9fbb714-953d-4ad8-90fb-6abb50788bd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:22:45.839482 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:22:45.899248 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4329c0a7-2e3c-4d34-a3ca-91764d570cd6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e48cba3f804cf08b2477f01ae95179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:22:45.900076 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4329c0a7-2e3c-4d34-a3ca-91764d570cd6 service nova] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:22:45.900385 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4329c0a7-2e3c-4d34-a3ca-91764d570cd6 service nova] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:22:45.900653 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4329c0a7-2e3c-4d34-a3ca-91764d570cd6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:22:45.902472 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-4329c0a7-2e3c-4d34-a3ca-91764d570cd6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:22:45.948250 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 535/1075] 10.222.0.18 () {58 vars in 1040 bytes} [Wed Nov 2 07:22:45 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:22:45.958143 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fbb714-953d-4ad8-90fb-6abb50788bd3 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:22:45.969590 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fbb714-953d-4ad8-90fb-6abb50788bd3 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:22:46.004758 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:22:46.015352 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4329c0a7-2e3c-4d34-a3ca-91764d570cd6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:22:46.017303 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4329c0a7-2e3c-4d34-a3ca-91764d570cd6 service nova] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:22:46.035745 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 541/1076] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:22:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 07:22:46.096559 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cee900-bb67-4b1b-aedd-ad7b28e1eedd service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9116f47889f4db39e0203d6cdae1593', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:22:46.097234 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cee900-bb67-4b1b-aedd-ad7b28e1eedd service swift] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:22:46.097570 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cee900-bb67-4b1b-aedd-ad7b28e1eedd service swift] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:22:46.097865 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cee900-bb67-4b1b-aedd-ad7b28e1eedd service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:22:46.099513 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-14cee900-bb67-4b1b-aedd-ad7b28e1eedd service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:22:46.205020 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cee900-bb67-4b1b-aedd-ad7b28e1eedd service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:22:46.206681 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cee900-bb67-4b1b-aedd-ad7b28e1eedd service swift] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:22:46.216390 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 536/1077] 10.222.0.18 () {60 vars in 1384 bytes} [Wed Nov 2 07:22:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 259 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Nov 02 07:22:48.251649 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f15203-8527-4dc8-89cf-f2dc98ccd66a None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:22:48.252195 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f15203-8527-4dc8-89cf-f2dc98ccd66a None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:22:48.252195 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f15203-8527-4dc8-89cf-f2dc98ccd66a None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:22:48.253516 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 542/1078] 10.222.0.18 () {54 vars in 927 bytes} [Wed Nov 2 07:22:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:22:48.258765 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1cff82-be3f-4815-947b-af6f86d604ea None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:22:48.259993 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1cff82-be3f-4815-947b-af6f86d604ea None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:22:48.259993 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1cff82-be3f-4815-947b-af6f86d604ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:22:48.334755 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-0a1cff82-be3f-4815-947b-af6f86d604ea None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:22:48.336140 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-0a1cff82-be3f-4815-947b-af6f86d604ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:22:48.345033 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:22:48.511602 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 537/1079] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Nov 2 07:22:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:22:48.807656 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f15203-8527-4dc8-89cf-f2dc98ccd66a None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:22:48.809150 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-46f15203-8527-4dc8-89cf-f2dc98ccd66a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:22:48.880926 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:22:48.972495 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f15203-8527-4dc8-89cf-f2dc98ccd66a None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:22:49.109848 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055d084d-5ea5-4099-b68e-38974c858979 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e48cba3f804cf08b2477f01ae95179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:22:49.110716 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055d084d-5ea5-4099-b68e-38974c858979 service nova] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:22:49.111094 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055d084d-5ea5-4099-b68e-38974c858979 service nova] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:22:49.111525 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055d084d-5ea5-4099-b68e-38974c858979 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:22:49.116325 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-055d084d-5ea5-4099-b68e-38974c858979 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:22:49.147564 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:22:49.222246 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055d084d-5ea5-4099-b68e-38974c858979 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:22:49.223926 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055d084d-5ea5-4099-b68e-38974c858979 service nova] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:22:49.242998 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 543/1080] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:22:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:22:51.470919 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328663b8-e581-4800-94aa-91737ced77db None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:22:51.471446 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328663b8-e581-4800-94aa-91737ced77db None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:22:51.471446 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328663b8-e581-4800-94aa-91737ced77db None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:22:51.472883 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 538/1081] 10.222.0.18 () {54 vars in 927 bytes} [Wed Nov 2 07:22:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:22:51.478198 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec6fc3c-671e-482d-b5fb-3c087d3778f4 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:22:51.478744 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec6fc3c-671e-482d-b5fb-3c087d3778f4 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:22:51.478894 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec6fc3c-671e-482d-b5fb-3c087d3778f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:22:51.556645 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-2ec6fc3c-671e-482d-b5fb-3c087d3778f4 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:22:51.557955 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-2ec6fc3c-671e-482d-b5fb-3c087d3778f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:22:51.678413 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 544/1082] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Nov 2 07:22:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:22:51.964308 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328663b8-e581-4800-94aa-91737ced77db None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:22:51.965815 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-328663b8-e581-4800-94aa-91737ced77db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:22:52.116190 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:22:52.127741 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328663b8-e581-4800-94aa-91737ced77db None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:22:52.254856 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcc5a69-d708-4607-8853-43b999a226aa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e48cba3f804cf08b2477f01ae95179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:22:52.255638 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcc5a69-d708-4607-8853-43b999a226aa service nova] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:22:52.255919 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcc5a69-d708-4607-8853-43b999a226aa service nova] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:22:52.256192 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcc5a69-d708-4607-8853-43b999a226aa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:22:52.257808 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-5bcc5a69-d708-4607-8853-43b999a226aa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:22:52.357131 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcc5a69-d708-4607-8853-43b999a226aa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:22:52.358751 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcc5a69-d708-4607-8853-43b999a226aa service nova] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:22:52.373213 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:22:52.383154 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 539/1083] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:22:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:22:54.608068 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e148589d-84d1-49ba-9474-794fdbd9a3ae None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:22:54.608641 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e148589d-84d1-49ba-9474-794fdbd9a3ae None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:22:54.608641 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e148589d-84d1-49ba-9474-794fdbd9a3ae None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:22:54.610143 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 545/1084] 10.222.0.18 () {54 vars in 927 bytes} [Wed Nov 2 07:22:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:22:54.615729 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ae8ff6-592a-44ed-8c09-917b388251b1 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:22:54.616185 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ae8ff6-592a-44ed-8c09-917b388251b1 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:22:54.616268 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ae8ff6-592a-44ed-8c09-917b388251b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:22:54.698863 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-a8ae8ff6-592a-44ed-8c09-917b388251b1 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:22:54.700099 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-a8ae8ff6-592a-44ed-8c09-917b388251b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:22:54.818523 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 540/1085] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Nov 2 07:22:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:22:55.102338 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e148589d-84d1-49ba-9474-794fdbd9a3ae None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:22:55.104962 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-e148589d-84d1-49ba-9474-794fdbd9a3ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:22:55.108033 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:22:55.270333 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e148589d-84d1-49ba-9474-794fdbd9a3ae None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:22:55.352134 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:22:55.402923 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1695da-5e4b-4639-9f89-c722f0d9f01c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e48cba3f804cf08b2477f01ae95179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:22:55.403526 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1695da-5e4b-4639-9f89-c722f0d9f01c service nova] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:22:55.403812 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1695da-5e4b-4639-9f89-c722f0d9f01c service nova] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:22:55.404083 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1695da-5e4b-4639-9f89-c722f0d9f01c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:22:55.405833 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-0e1695da-5e4b-4639-9f89-c722f0d9f01c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:22:55.507508 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1695da-5e4b-4639-9f89-c722f0d9f01c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:22:55.509870 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1695da-5e4b-4639-9f89-c722f0d9f01c service nova] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:22:55.529400 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 546/1086] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:22:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:23:04.808073 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=67 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:23:05.527138 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=61 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:23:09.074598 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c07e46-09d0-4102-9e0e-27b9c04b3d00 None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:23:09.075175 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c07e46-09d0-4102-9e0e-27b9c04b3d00 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:23:09.075175 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c07e46-09d0-4102-9e0e-27b9c04b3d00 None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:23:09.076484 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 541/1087] 10.222.0.18 () {54 vars in 918 bytes} [Wed Nov 2 07:23:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:23:09.081774 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3aa7979-bd6d-4c88-b2b6-1b944da8e8d0 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:23:09.082208 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3aa7979-bd6d-4c88-b2b6-1b944da8e8d0 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:23:09.082309 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3aa7979-bd6d-4c88-b2b6-1b944da8e8d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:23:09.158220 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-d3aa7979-bd6d-4c88-b2b6-1b944da8e8d0 None None] MFA Rules not processed for user `4f2fc6a156c54d0c9d325901d0f7f6fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:23:09.159444 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-d3aa7979-bd6d-4c88-b2b6-1b944da8e8d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:23:09.264145 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 547/1088] 10.222.0.18 () {58 vars in 1015 bytes} [Wed Nov 2 07:23:09 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:23:09.278598 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537fcfa1-2580-4273-8641-caf024f6c981 None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:23:09.278884 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537fcfa1-2580-4273-8641-caf024f6c981 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:23:09.279155 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537fcfa1-2580-4273-8641-caf024f6c981 None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:23:09.280471 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 542/1089] 10.222.0.18 () {54 vars in 918 bytes} [Wed Nov 2 07:23:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:23:09.285130 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c565bd55-d8c9-4e78-8523-4c6f3e52ad6a None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:23:09.285415 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c565bd55-d8c9-4e78-8523-4c6f3e52ad6a None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:23:09.285685 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c565bd55-d8c9-4e78-8523-4c6f3e52ad6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:23:09.357801 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:23:09.361824 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-c565bd55-d8c9-4e78-8523-4c6f3e52ad6a None None] MFA Rules not processed for user `69e48cba3f804cf08b2477f01ae95179`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:23:09.363568 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-c565bd55-d8c9-4e78-8523-4c6f3e52ad6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:23:09.477636 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 548/1090] 10.222.0.18 () {58 vars in 1015 bytes} [Wed Nov 2 07:23:09 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:23:12.055153 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae457fe8-2189-48f8-877c-a35fec3724b4 None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:23:12.055725 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae457fe8-2189-48f8-877c-a35fec3724b4 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:23:12.055725 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae457fe8-2189-48f8-877c-a35fec3724b4 None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:23:12.057076 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 543/1091] 127.0.0.1 () {54 vars in 917 bytes} [Wed Nov 2 07:23:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:23:12.062488 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ac4a6d-329d-404c-ae82-94c5cef6aed3 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:23:12.062916 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ac4a6d-329d-404c-ae82-94c5cef6aed3 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:23:12.063035 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ac4a6d-329d-404c-ae82-94c5cef6aed3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:23:12.128054 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-47ac4a6d-329d-404c-ae82-94c5cef6aed3 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:23:12.129252 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-47ac4a6d-329d-404c-ae82-94c5cef6aed3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:23:12.184726 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:23:12.227383 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 549/1092] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Nov 2 07:23:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:23:12.523355 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae457fe8-2189-48f8-877c-a35fec3724b4 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:23:12.524980 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-ae457fe8-2189-48f8-877c-a35fec3724b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:23:12.670399 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae457fe8-2189-48f8-877c-a35fec3724b4 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:23:12.741335 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:23:12.781178 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480b6039-9bad-4fc3-9005-19cd37ca3c22 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e48cba3f804cf08b2477f01ae95179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:23:12.781712 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480b6039-9bad-4fc3-9005-19cd37ca3c22 service nova] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:23:12.781991 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480b6039-9bad-4fc3-9005-19cd37ca3c22 service nova] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:23:12.782261 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480b6039-9bad-4fc3-9005-19cd37ca3c22 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:23:12.783868 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-480b6039-9bad-4fc3-9005-19cd37ca3c22 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:23:12.874065 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480b6039-9bad-4fc3-9005-19cd37ca3c22 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:23:12.875667 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480b6039-9bad-4fc3-9005-19cd37ca3c22 service nova] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:23:12.893840 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 544/1093] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:23:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:23:14.021212 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f661ea6-1015-41bb-bf61-8377fb83bac0 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:23:14.021758 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f661ea6-1015-41bb-bf61-8377fb83bac0 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:23:14.021758 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f661ea6-1015-41bb-bf61-8377fb83bac0 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:23:14.023130 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 550/1094] 127.0.0.1 () {54 vars in 901 bytes} [Wed Nov 2 07:23:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:23:14.028525 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfae525-f0b3-41d5-852b-dd792c2f22c5 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:23:14.028988 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfae525-f0b3-41d5-852b-dd792c2f22c5 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:23:14.029093 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfae525-f0b3-41d5-852b-dd792c2f22c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:23:14.104328 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-edfae525-f0b3-41d5-852b-dd792c2f22c5 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:23:14.105513 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-edfae525-f0b3-41d5-852b-dd792c2f22c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:23:14.132639 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:23:14.201042 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 545/1095] 127.0.0.1 () {58 vars in 998 bytes} [Wed Nov 2 07:23:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:23:14.367938 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f661ea6-1015-41bb-bf61-8377fb83bac0 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:23:14.369500 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-2f661ea6-1015-41bb-bf61-8377fb83bac0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:23:14.517564 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f661ea6-1015-41bb-bf61-8377fb83bac0 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:23:14.558039 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:23:14.630169 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4858fe-b862-4c61-b8e2-b38c893fe97c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e48cba3f804cf08b2477f01ae95179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:23:14.630838 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4858fe-b862-4c61-b8e2-b38c893fe97c service nova] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:23:14.631115 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4858fe-b862-4c61-b8e2-b38c893fe97c service nova] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:23:14.631381 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4858fe-b862-4c61-b8e2-b38c893fe97c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:23:14.632945 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-ba4858fe-b862-4c61-b8e2-b38c893fe97c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:23:14.723235 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4858fe-b862-4c61-b8e2-b38c893fe97c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:23:14.724856 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4858fe-b862-4c61-b8e2-b38c893fe97c service nova] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:23:14.743223 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 551/1096] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:23:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:23:15.853679 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829d971a-4249-4a41-ace4-90daf04a6e86 None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:23:15.854251 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829d971a-4249-4a41-ace4-90daf04a6e86 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:23:15.854251 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829d971a-4249-4a41-ace4-90daf04a6e86 None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:23:15.855660 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 546/1097] 127.0.0.1 () {54 vars in 901 bytes} [Wed Nov 2 07:23:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:23:15.860687 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c54a03-a61f-4bbb-bda4-85542a54f56e None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:23:15.861097 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c54a03-a61f-4bbb-bda4-85542a54f56e None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:23:15.861233 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c54a03-a61f-4bbb-bda4-85542a54f56e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:23:15.915373 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:23:15.936804 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-66c54a03-a61f-4bbb-bda4-85542a54f56e None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:23:15.938001 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-66c54a03-a61f-4bbb-bda4-85542a54f56e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:23:16.033157 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 552/1098] 127.0.0.1 () {58 vars in 998 bytes} [Wed Nov 2 07:23:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:23:16.184080 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829d971a-4249-4a41-ace4-90daf04a6e86 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:23:16.185477 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-829d971a-4249-4a41-ace4-90daf04a6e86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:23:16.335334 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829d971a-4249-4a41-ace4-90daf04a6e86 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:23:16.347429 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:23:16.448574 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6d5aab-8f91-4b99-8c03-249513402169 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e48cba3f804cf08b2477f01ae95179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:23:16.449642 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6d5aab-8f91-4b99-8c03-249513402169 service nova] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:23:16.449919 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6d5aab-8f91-4b99-8c03-249513402169 service nova] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:23:16.450187 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6d5aab-8f91-4b99-8c03-249513402169 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:23:16.451803 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-6d6d5aab-8f91-4b99-8c03-249513402169 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:23:16.539806 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6d5aab-8f91-4b99-8c03-249513402169 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:23:16.541379 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6d5aab-8f91-4b99-8c03-249513402169 service nova] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:23:16.559973 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 547/1099] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:23:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:23:17.658838 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0437b19-9ba8-48c4-9e30-63c07dc90f0a None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:23:17.659332 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0437b19-9ba8-48c4-9e30-63c07dc90f0a None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:23:17.659430 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0437b19-9ba8-48c4-9e30-63c07dc90f0a None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:23:17.660767 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 553/1100] 127.0.0.1 () {54 vars in 901 bytes} [Wed Nov 2 07:23:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:23:17.665811 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e74505-3fb8-46ba-891f-9d3987948549 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:23:17.666238 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e74505-3fb8-46ba-891f-9d3987948549 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:23:17.666345 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e74505-3fb8-46ba-891f-9d3987948549 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:23:17.686224 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:23:17.743034 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-08e74505-3fb8-46ba-891f-9d3987948549 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:23:17.744233 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-08e74505-3fb8-46ba-891f-9d3987948549 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:23:17.840036 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 548/1101] 127.0.0.1 () {58 vars in 998 bytes} [Wed Nov 2 07:23:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:23:17.990488 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0437b19-9ba8-48c4-9e30-63c07dc90f0a None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:23:17.992160 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-e0437b19-9ba8-48c4-9e30-63c07dc90f0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:23:18.099822 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:23:18.136419 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0437b19-9ba8-48c4-9e30-63c07dc90f0a None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:23:18.248990 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457b6ab2-c6ff-4659-b72c-22df8b8e8f3e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e48cba3f804cf08b2477f01ae95179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:23:18.249596 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457b6ab2-c6ff-4659-b72c-22df8b8e8f3e service nova] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:23:18.249870 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457b6ab2-c6ff-4659-b72c-22df8b8e8f3e service nova] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:23:18.250135 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457b6ab2-c6ff-4659-b72c-22df8b8e8f3e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:23:18.251732 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-457b6ab2-c6ff-4659-b72c-22df8b8e8f3e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:23:18.328924 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:23:18.343224 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457b6ab2-c6ff-4659-b72c-22df8b8e8f3e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:23:18.344819 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457b6ab2-c6ff-4659-b72c-22df8b8e8f3e service nova] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:23:18.363485 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 554/1102] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:23:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:23:19.461157 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd9d6b8-0d34-457a-a919-544cd126675e None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:23:19.461672 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd9d6b8-0d34-457a-a919-544cd126675e None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:23:19.461672 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd9d6b8-0d34-457a-a919-544cd126675e None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:23:19.462988 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 549/1103] 127.0.0.1 () {54 vars in 901 bytes} [Wed Nov 2 07:23:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:23:19.468556 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23033bf9-cf84-4617-bb2e-2dc46fddc888 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:23:19.468979 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23033bf9-cf84-4617-bb2e-2dc46fddc888 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:23:19.469074 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23033bf9-cf84-4617-bb2e-2dc46fddc888 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:23:19.544412 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-23033bf9-cf84-4617-bb2e-2dc46fddc888 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:23:19.545602 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-23033bf9-cf84-4617-bb2e-2dc46fddc888 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:23:19.640137 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 555/1104] 127.0.0.1 () {58 vars in 998 bytes} [Wed Nov 2 07:23:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:23:19.789850 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd9d6b8-0d34-457a-a919-544cd126675e None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:23:19.791388 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-bbd9d6b8-0d34-457a-a919-544cd126675e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:23:19.876869 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:23:19.938478 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd9d6b8-0d34-457a-a919-544cd126675e None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:23:20.049322 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77f187c-b9f4-41bb-8c72-f51aa374a51d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e48cba3f804cf08b2477f01ae95179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:23:20.049841 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77f187c-b9f4-41bb-8c72-f51aa374a51d service nova] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:23:20.050119 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77f187c-b9f4-41bb-8c72-f51aa374a51d service nova] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:23:20.050416 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77f187c-b9f4-41bb-8c72-f51aa374a51d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:23:20.051984 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-b77f187c-b9f4-41bb-8c72-f51aa374a51d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:23:20.099572 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:23:20.141993 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77f187c-b9f4-41bb-8c72-f51aa374a51d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:23:20.143609 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77f187c-b9f4-41bb-8c72-f51aa374a51d service nova] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:23:20.161585 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 550/1105] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:23:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:23:21.210110 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a91d6a-1ef4-478e-a92c-a1b69f6d0eee None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:23:21.210537 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a91d6a-1ef4-478e-a92c-a1b69f6d0eee None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:23:21.210959 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a91d6a-1ef4-478e-a92c-a1b69f6d0eee None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:23:21.212031 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 556/1106] 127.0.0.1 () {54 vars in 901 bytes} [Wed Nov 2 07:23:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:23:21.217935 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47eab7c-51c4-4017-9006-63a0161c7d6f None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:23:21.218448 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47eab7c-51c4-4017-9006-63a0161c7d6f None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:23:21.218547 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47eab7c-51c4-4017-9006-63a0161c7d6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:23:21.296161 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-e47eab7c-51c4-4017-9006-63a0161c7d6f None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:23:21.297351 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-e47eab7c-51c4-4017-9006-63a0161c7d6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:23:21.399002 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 551/1107] 127.0.0.1 () {58 vars in 998 bytes} [Wed Nov 2 07:23:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:23:21.550537 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a91d6a-1ef4-478e-a92c-a1b69f6d0eee None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:23:21.552083 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-98a91d6a-1ef4-478e-a92c-a1b69f6d0eee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:23:21.600662 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:23:21.698567 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a91d6a-1ef4-478e-a92c-a1b69f6d0eee None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:23:21.811386 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e639bd2e-ec4f-4e3b-a6f4-fe54e3db8afb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e48cba3f804cf08b2477f01ae95179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:23:21.811904 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e639bd2e-ec4f-4e3b-a6f4-fe54e3db8afb service nova] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:23:21.812179 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e639bd2e-ec4f-4e3b-a6f4-fe54e3db8afb service nova] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:23:21.812444 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e639bd2e-ec4f-4e3b-a6f4-fe54e3db8afb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:23:21.813993 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-e639bd2e-ec4f-4e3b-a6f4-fe54e3db8afb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:23:21.820887 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:23:21.904510 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e639bd2e-ec4f-4e3b-a6f4-fe54e3db8afb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:23:21.906088 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e639bd2e-ec4f-4e3b-a6f4-fe54e3db8afb service nova] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:23:21.923962 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 557/1108] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:23:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:23:22.957065 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fc1161-e9d2-4d3f-bc39-4a586cd22e4e None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:23:22.957584 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fc1161-e9d2-4d3f-bc39-4a586cd22e4e None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:23:22.957584 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fc1161-e9d2-4d3f-bc39-4a586cd22e4e None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:23:22.958959 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 552/1109] 127.0.0.1 () {54 vars in 901 bytes} [Wed Nov 2 07:23:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:23:22.963992 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f485dfd-0747-4ae0-8413-466e1839e8d2 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:23:22.964430 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f485dfd-0747-4ae0-8413-466e1839e8d2 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:23:22.965097 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f485dfd-0747-4ae0-8413-466e1839e8d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:23:23.039497 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-3f485dfd-0747-4ae0-8413-466e1839e8d2 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:23:23.040698 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-3f485dfd-0747-4ae0-8413-466e1839e8d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:23:23.107888 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:23:23.135914 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 558/1110] 127.0.0.1 () {58 vars in 998 bytes} [Wed Nov 2 07:23:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:23:23.288332 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fc1161-e9d2-4d3f-bc39-4a586cd22e4e None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:23:23.289835 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-34fc1161-e9d2-4d3f-bc39-4a586cd22e4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:23:23.307177 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:23:23.436124 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fc1161-e9d2-4d3f-bc39-4a586cd22e4e None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:23:23.524179 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:23:23.550908 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208f5caf-7cdd-4103-be91-7b7f34c0f9d9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e48cba3f804cf08b2477f01ae95179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:23:23.551427 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208f5caf-7cdd-4103-be91-7b7f34c0f9d9 service nova] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:23:23.551709 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208f5caf-7cdd-4103-be91-7b7f34c0f9d9 service nova] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:23:23.551980 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208f5caf-7cdd-4103-be91-7b7f34c0f9d9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:23:23.553571 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-208f5caf-7cdd-4103-be91-7b7f34c0f9d9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:23:23.642501 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208f5caf-7cdd-4103-be91-7b7f34c0f9d9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:23:23.644102 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208f5caf-7cdd-4103-be91-7b7f34c0f9d9 service nova] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:23:23.662596 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 553/1111] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:23:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:23:24.706747 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e8aa95-848e-4b61-8011-d9e7be5fc0ef None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:23:24.707301 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e8aa95-848e-4b61-8011-d9e7be5fc0ef None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:23:24.707301 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e8aa95-848e-4b61-8011-d9e7be5fc0ef None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:23:24.708636 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 559/1112] 127.0.0.1 () {54 vars in 901 bytes} [Wed Nov 2 07:23:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:23:24.713414 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51109daf-2d92-455c-9df8-9411965ec696 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:23:24.713823 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51109daf-2d92-455c-9df8-9411965ec696 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:23:24.713955 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51109daf-2d92-455c-9df8-9411965ec696 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:23:24.788834 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-51109daf-2d92-455c-9df8-9411965ec696 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:23:24.790026 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-51109daf-2d92-455c-9df8-9411965ec696 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:23:24.830343 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:23:24.884583 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 554/1113] 127.0.0.1 () {58 vars in 998 bytes} [Wed Nov 2 07:23:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:23:25.036677 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e8aa95-848e-4b61-8011-d9e7be5fc0ef None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:23:25.038162 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-e2e8aa95-848e-4b61-8011-d9e7be5fc0ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:23:25.182811 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e8aa95-848e-4b61-8011-d9e7be5fc0ef None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:23:25.237734 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:23:25.295658 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c01a20-f4e2-43d0-9c96-fc71f69a8901 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e48cba3f804cf08b2477f01ae95179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:23:25.296255 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c01a20-f4e2-43d0-9c96-fc71f69a8901 service nova] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:23:25.296530 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c01a20-f4e2-43d0-9c96-fc71f69a8901 service nova] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:23:25.296796 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c01a20-f4e2-43d0-9c96-fc71f69a8901 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:23:25.298352 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-90c01a20-f4e2-43d0-9c96-fc71f69a8901 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:23:25.387020 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c01a20-f4e2-43d0-9c96-fc71f69a8901 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:23:25.388702 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c01a20-f4e2-43d0-9c96-fc71f69a8901 service nova] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:23:25.406892 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 560/1114] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:23:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:23:26.446694 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea23c535-4d08-4cb9-aed8-f681697d9320 None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:23:26.447286 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea23c535-4d08-4cb9-aed8-f681697d9320 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:23:26.447286 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea23c535-4d08-4cb9-aed8-f681697d9320 None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:23:26.448589 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 555/1115] 127.0.0.1 () {54 vars in 901 bytes} [Wed Nov 2 07:23:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:23:26.453909 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c7b551-85c1-4650-af50-17b81efec46a None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:23:26.454339 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c7b551-85c1-4650-af50-17b81efec46a None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:23:26.454464 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c7b551-85c1-4650-af50-17b81efec46a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:23:26.529193 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-73c7b551-85c1-4650-af50-17b81efec46a None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:23:26.530411 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-73c7b551-85c1-4650-af50-17b81efec46a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:23:26.532410 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:23:26.627008 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 561/1116] 127.0.0.1 () {58 vars in 998 bytes} [Wed Nov 2 07:23:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:23:26.775564 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea23c535-4d08-4cb9-aed8-f681697d9320 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:23:26.777048 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-ea23c535-4d08-4cb9-aed8-f681697d9320 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:23:26.919424 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea23c535-4d08-4cb9-aed8-f681697d9320 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:23:26.947659 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:23:27.033151 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f42e01-7065-4983-b3fd-6c37c960f0c7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e48cba3f804cf08b2477f01ae95179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:23:27.033672 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f42e01-7065-4983-b3fd-6c37c960f0c7 service nova] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:23:27.033949 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f42e01-7065-4983-b3fd-6c37c960f0c7 service nova] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:23:27.034216 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f42e01-7065-4983-b3fd-6c37c960f0c7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:23:27.035831 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-d2f42e01-7065-4983-b3fd-6c37c960f0c7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:23:27.123019 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f42e01-7065-4983-b3fd-6c37c960f0c7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:23:27.124610 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f42e01-7065-4983-b3fd-6c37c960f0c7 service nova] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:23:27.142919 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 556/1117] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:23:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 07:23:29.204862 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b635c3d1-9a7a-4bbb-bbb8-b8d3d83fbc3a None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:23:29.205335 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b635c3d1-9a7a-4bbb-bbb8-b8d3d83fbc3a None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:23:29.205444 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b635c3d1-9a7a-4bbb-bbb8-b8d3d83fbc3a None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:23:29.206769 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 562/1118] 127.0.0.1 () {54 vars in 917 bytes} [Wed Nov 2 07:23:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:23:29.212135 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec189b2e-382f-4e3f-ba65-c149a4b9c6fe None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:23:29.212529 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec189b2e-382f-4e3f-ba65-c149a4b9c6fe None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:23:29.212685 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec189b2e-382f-4e3f-ba65-c149a4b9c6fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:23:29.254253 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:23:29.279934 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-ec189b2e-382f-4e3f-ba65-c149a4b9c6fe None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:23:29.281122 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-ec189b2e-382f-4e3f-ba65-c149a4b9c6fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:23:29.377871 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 557/1119] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Nov 2 07:23:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:23:29.664665 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b635c3d1-9a7a-4bbb-bbb8-b8d3d83fbc3a None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:23:29.666184 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-b635c3d1-9a7a-4bbb-bbb8-b8d3d83fbc3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:23:29.768441 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b635c3d1-9a7a-4bbb-bbb8-b8d3d83fbc3a None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:23:29.801892 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:23:29.843695 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe12fb5c-42e8-4324-b988-2ea177ec89f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3ec6ecb85f94ef59762877d9ea33ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:23:29.844292 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe12fb5c-42e8-4324-b988-2ea177ec89f6 service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:23:29.844564 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe12fb5c-42e8-4324-b988-2ea177ec89f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:23:29.844826 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe12fb5c-42e8-4324-b988-2ea177ec89f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:23:29.846410 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-fe12fb5c-42e8-4324-b988-2ea177ec89f6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:23:29.952288 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe12fb5c-42e8-4324-b988-2ea177ec89f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:23:29.953630 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe12fb5c-42e8-4324-b988-2ea177ec89f6 service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:23:29.971639 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 563/1120] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:23:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:23:32.270884 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fdd78a-af5e-42ea-8125-751380cf1e1c None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:23:32.271487 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fdd78a-af5e-42ea-8125-751380cf1e1c None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:23:32.271487 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fdd78a-af5e-42ea-8125-751380cf1e1c None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:23:32.272828 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 558/1121] 127.0.0.1 () {54 vars in 917 bytes} [Wed Nov 2 07:23:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:23:32.278846 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7ff593-571b-449e-81b9-5a799d8fc3f9 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:23:32.279261 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7ff593-571b-449e-81b9-5a799d8fc3f9 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:23:32.279395 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7ff593-571b-449e-81b9-5a799d8fc3f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:23:32.345338 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-8c7ff593-571b-449e-81b9-5a799d8fc3f9 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:23:32.346564 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-8c7ff593-571b-449e-81b9-5a799d8fc3f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:23:32.359697 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:23:32.445447 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 564/1122] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Nov 2 07:23:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:23:32.730202 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fdd78a-af5e-42ea-8125-751380cf1e1c None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:23:32.731953 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-a7fdd78a-af5e-42ea-8125-751380cf1e1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:23:32.834538 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fdd78a-af5e-42ea-8125-751380cf1e1c None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:23:32.847181 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:23:32.907818 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e1cf53-fb68-4fc4-9f39-af1be4b03611 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3ec6ecb85f94ef59762877d9ea33ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:23:32.908357 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e1cf53-fb68-4fc4-9f39-af1be4b03611 service neutron] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:23:32.908631 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e1cf53-fb68-4fc4-9f39-af1be4b03611 service neutron] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:23:32.908895 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e1cf53-fb68-4fc4-9f39-af1be4b03611 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:23:32.910482 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-b0e1cf53-fb68-4fc4-9f39-af1be4b03611 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:23:33.019130 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e1cf53-fb68-4fc4-9f39-af1be4b03611 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:23:33.020540 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e1cf53-fb68-4fc4-9f39-af1be4b03611 service neutron] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:23:33.037787 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 559/1123] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:23:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:23:35.486833 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ec5e90-cbb0-4a60-8740-2bada1a140ff None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:23:35.487373 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ec5e90-cbb0-4a60-8740-2bada1a140ff None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:23:35.487373 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ec5e90-cbb0-4a60-8740-2bada1a140ff None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:23:35.488669 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 565/1124] 127.0.0.1 () {54 vars in 917 bytes} [Wed Nov 2 07:23:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:23:35.494257 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ff4404-4bcf-426f-8d64-8615043a08db None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:23:35.494743 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ff4404-4bcf-426f-8d64-8615043a08db None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:23:35.494849 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ff4404-4bcf-426f-8d64-8615043a08db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:23:35.542571 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:23:35.562183 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-24ff4404-4bcf-426f-8d64-8615043a08db None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:23:35.563471 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-24ff4404-4bcf-426f-8d64-8615043a08db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:23:35.660441 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 560/1125] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Nov 2 07:23:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:23:35.945233 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ec5e90-cbb0-4a60-8740-2bada1a140ff None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:23:35.946789 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-44ec5e90-cbb0-4a60-8740-2bada1a140ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:23:36.049902 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ec5e90-cbb0-4a60-8740-2bada1a140ff None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:23:36.090917 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:23:36.125589 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61c10e7-d2f2-4c2f-b5e5-a05beac24e33 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3ec6ecb85f94ef59762877d9ea33ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:23:36.126184 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61c10e7-d2f2-4c2f-b5e5-a05beac24e33 service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:23:36.126492 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61c10e7-d2f2-4c2f-b5e5-a05beac24e33 service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:23:36.126763 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61c10e7-d2f2-4c2f-b5e5-a05beac24e33 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:23:36.128322 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-c61c10e7-d2f2-4c2f-b5e5-a05beac24e33 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:23:36.233989 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61c10e7-d2f2-4c2f-b5e5-a05beac24e33 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:23:36.235332 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61c10e7-d2f2-4c2f-b5e5-a05beac24e33 service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:23:36.252859 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 566/1126] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:23:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:23:38.553363 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb0380a-edce-4c6f-81de-c922d868e143 None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:23:38.553949 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb0380a-edce-4c6f-81de-c922d868e143 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:23:38.553949 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb0380a-edce-4c6f-81de-c922d868e143 None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:23:38.555397 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 561/1127] 127.0.0.1 () {54 vars in 917 bytes} [Wed Nov 2 07:23:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:23:38.561337 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc827830-68f7-4454-8b1d-7bd92a42e11f None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:23:38.561751 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc827830-68f7-4454-8b1d-7bd92a42e11f None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:23:38.561942 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc827830-68f7-4454-8b1d-7bd92a42e11f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:23:38.627716 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-dc827830-68f7-4454-8b1d-7bd92a42e11f None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:23:38.628910 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-dc827830-68f7-4454-8b1d-7bd92a42e11f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:23:38.649798 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:23:38.726218 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 567/1128] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Nov 2 07:23:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:23:39.009140 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb0380a-edce-4c6f-81de-c922d868e143 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:23:39.010678 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-0eb0380a-edce-4c6f-81de-c922d868e143 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:23:39.113671 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb0380a-edce-4c6f-81de-c922d868e143 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:23:39.134810 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:23:39.188057 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d067fc6-4995-4ce2-ad96-cf1c07ce7f44 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3ec6ecb85f94ef59762877d9ea33ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:23:39.188648 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d067fc6-4995-4ce2-ad96-cf1c07ce7f44 service neutron] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:23:39.188924 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d067fc6-4995-4ce2-ad96-cf1c07ce7f44 service neutron] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:23:39.189190 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d067fc6-4995-4ce2-ad96-cf1c07ce7f44 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:23:39.190878 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-4d067fc6-4995-4ce2-ad96-cf1c07ce7f44 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:23:39.298238 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d067fc6-4995-4ce2-ad96-cf1c07ce7f44 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:23:39.299624 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d067fc6-4995-4ce2-ad96-cf1c07ce7f44 service neutron] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:23:39.317745 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 562/1129] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:23:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:23:41.537810 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387ea187-3af8-46f7-8916-33793c240d99 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:23:41.538382 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387ea187-3af8-46f7-8916-33793c240d99 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:23:41.538484 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387ea187-3af8-46f7-8916-33793c240d99 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:23:41.539751 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 568/1130] 127.0.0.1 () {54 vars in 917 bytes} [Wed Nov 2 07:23:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:23:41.545801 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de52d398-19b5-4b33-ac50-4dcbcfc2f536 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:23:41.546234 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de52d398-19b5-4b33-ac50-4dcbcfc2f536 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:23:41.546353 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de52d398-19b5-4b33-ac50-4dcbcfc2f536 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:23:41.610565 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:23:41.613329 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-de52d398-19b5-4b33-ac50-4dcbcfc2f536 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:23:41.615085 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-de52d398-19b5-4b33-ac50-4dcbcfc2f536 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:23:41.711634 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 563/1131] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Nov 2 07:23:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:23:41.997638 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387ea187-3af8-46f7-8916-33793c240d99 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:23:41.999225 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-387ea187-3af8-46f7-8916-33793c240d99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:23:42.100365 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387ea187-3af8-46f7-8916-33793c240d99 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:23:42.150613 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:23:42.175149 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccbae55-cab4-4c45-b183-3afdb1e83eed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3ec6ecb85f94ef59762877d9ea33ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:23:42.175741 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccbae55-cab4-4c45-b183-3afdb1e83eed service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:23:42.176013 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccbae55-cab4-4c45-b183-3afdb1e83eed service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:23:42.176273 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccbae55-cab4-4c45-b183-3afdb1e83eed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:23:42.177819 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-cccbae55-cab4-4c45-b183-3afdb1e83eed service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:23:42.285393 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccbae55-cab4-4c45-b183-3afdb1e83eed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:23:42.286852 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccbae55-cab4-4c45-b183-3afdb1e83eed service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:23:42.304732 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 569/1132] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:23:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:23:44.591277 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bebc3af-07ab-43f3-a412-92d7b6d415de None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:23:44.591853 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bebc3af-07ab-43f3-a412-92d7b6d415de None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:23:44.591853 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bebc3af-07ab-43f3-a412-92d7b6d415de None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:23:44.593159 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 564/1133] 127.0.0.1 () {54 vars in 917 bytes} [Wed Nov 2 07:23:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:23:44.598648 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bc7fbc-a42c-4818-8d0e-2a29cd73ffe1 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:23:44.599082 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bc7fbc-a42c-4818-8d0e-2a29cd73ffe1 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:23:44.599201 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bc7fbc-a42c-4818-8d0e-2a29cd73ffe1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:23:44.666973 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-70bc7fbc-a42c-4818-8d0e-2a29cd73ffe1 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:23:44.668194 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-70bc7fbc-a42c-4818-8d0e-2a29cd73ffe1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:23:44.700842 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:23:44.767191 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 570/1134] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Nov 2 07:23:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:23:45.050686 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bebc3af-07ab-43f3-a412-92d7b6d415de None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:23:45.052209 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-6bebc3af-07ab-43f3-a412-92d7b6d415de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:23:45.155516 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bebc3af-07ab-43f3-a412-92d7b6d415de None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:23:45.183298 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:23:45.228850 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4637ead6-05d2-451c-b62c-96d0c8acfb94 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3ec6ecb85f94ef59762877d9ea33ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:23:45.229451 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4637ead6-05d2-451c-b62c-96d0c8acfb94 service neutron] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:23:45.229728 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4637ead6-05d2-451c-b62c-96d0c8acfb94 service neutron] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:23:45.229996 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4637ead6-05d2-451c-b62c-96d0c8acfb94 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:23:45.232106 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-4637ead6-05d2-451c-b62c-96d0c8acfb94 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:23:45.341364 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4637ead6-05d2-451c-b62c-96d0c8acfb94 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:23:45.342752 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4637ead6-05d2-451c-b62c-96d0c8acfb94 service neutron] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:23:45.361166 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 565/1135] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:23:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:23:47.571577 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef8a020-4905-4195-a5fd-840928437f39 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:23:47.572118 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef8a020-4905-4195-a5fd-840928437f39 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:23:47.572118 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef8a020-4905-4195-a5fd-840928437f39 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:23:47.573433 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 571/1136] 127.0.0.1 () {54 vars in 917 bytes} [Wed Nov 2 07:23:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:23:47.579105 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2aab4f-61f5-48e5-b11c-88636b004d2f None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:23:47.579554 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2aab4f-61f5-48e5-b11c-88636b004d2f None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:23:47.579671 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2aab4f-61f5-48e5-b11c-88636b004d2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:23:47.645264 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-6d2aab4f-61f5-48e5-b11c-88636b004d2f None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:23:47.646538 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-6d2aab4f-61f5-48e5-b11c-88636b004d2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:23:47.655701 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:23:47.744458 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 566/1137] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Nov 2 07:23:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:23:48.029035 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef8a020-4905-4195-a5fd-840928437f39 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:23:48.030579 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-eef8a020-4905-4195-a5fd-840928437f39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:23:48.131455 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef8a020-4905-4195-a5fd-840928437f39 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:23:48.185914 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:23:48.204315 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6536f95d-fe3e-4a33-b51c-8972d220ddd0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3ec6ecb85f94ef59762877d9ea33ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:23:48.204907 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6536f95d-fe3e-4a33-b51c-8972d220ddd0 service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:23:48.205183 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6536f95d-fe3e-4a33-b51c-8972d220ddd0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:23:48.205444 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6536f95d-fe3e-4a33-b51c-8972d220ddd0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:23:48.207022 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-6536f95d-fe3e-4a33-b51c-8972d220ddd0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:23:48.314590 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6536f95d-fe3e-4a33-b51c-8972d220ddd0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:23:48.315950 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6536f95d-fe3e-4a33-b51c-8972d220ddd0 service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:23:48.333907 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 572/1138] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:23:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:23:50.547371 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1ea483-5315-4d5b-b811-a747fe96119c None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:23:50.547938 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1ea483-5315-4d5b-b811-a747fe96119c None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:23:50.547938 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1ea483-5315-4d5b-b811-a747fe96119c None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:23:50.549278 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 567/1139] 127.0.0.1 () {54 vars in 917 bytes} [Wed Nov 2 07:23:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:23:50.554604 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba3bc1b-d35c-4bb1-bea4-578085778048 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:23:50.555019 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba3bc1b-d35c-4bb1-bea4-578085778048 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:23:50.555142 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba3bc1b-d35c-4bb1-bea4-578085778048 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:23:50.624145 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-0ba3bc1b-d35c-4bb1-bea4-578085778048 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:23:50.625356 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-0ba3bc1b-d35c-4bb1-bea4-578085778048 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:23:50.664051 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:23:50.722339 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 573/1140] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Nov 2 07:23:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:23:51.004886 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1ea483-5315-4d5b-b811-a747fe96119c None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:23:51.006484 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-0a1ea483-5315-4d5b-b811-a747fe96119c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:23:51.108046 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1ea483-5315-4d5b-b811-a747fe96119c None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:23:51.146050 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:23:51.181026 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd09260-8c16-46c8-bd64-cb4c089cf26e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3ec6ecb85f94ef59762877d9ea33ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:23:51.181623 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd09260-8c16-46c8-bd64-cb4c089cf26e service neutron] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:23:51.181904 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd09260-8c16-46c8-bd64-cb4c089cf26e service neutron] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:23:51.182172 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd09260-8c16-46c8-bd64-cb4c089cf26e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:23:51.183883 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-3fd09260-8c16-46c8-bd64-cb4c089cf26e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:23:51.294801 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd09260-8c16-46c8-bd64-cb4c089cf26e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:23:51.296421 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd09260-8c16-46c8-bd64-cb4c089cf26e service neutron] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:23:51.314926 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 568/1141] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:23:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:23:53.482816 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf3eb60-861b-47bb-9780-4c5f30afb385 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:23:53.483342 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf3eb60-861b-47bb-9780-4c5f30afb385 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:23:53.483342 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf3eb60-861b-47bb-9780-4c5f30afb385 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:23:53.484612 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 574/1142] 127.0.0.1 () {54 vars in 917 bytes} [Wed Nov 2 07:23:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:23:53.490136 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cef866-01bd-4136-b2ed-b10e4f6716f8 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:23:53.490591 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cef866-01bd-4136-b2ed-b10e4f6716f8 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:23:53.490718 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cef866-01bd-4136-b2ed-b10e4f6716f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:23:53.554962 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-84cef866-01bd-4136-b2ed-b10e4f6716f8 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:23:53.556152 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-84cef866-01bd-4136-b2ed-b10e4f6716f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:23:53.574438 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:23:53.651998 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 569/1143] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Nov 2 07:23:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:23:53.938384 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf3eb60-861b-47bb-9780-4c5f30afb385 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:23:53.939921 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-8cf3eb60-861b-47bb-9780-4c5f30afb385 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:23:54.041370 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf3eb60-861b-47bb-9780-4c5f30afb385 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:23:54.107856 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:23:54.114658 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a82bc36-4606-401b-90a1-c5b10d2efff7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3ec6ecb85f94ef59762877d9ea33ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:23:54.115257 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a82bc36-4606-401b-90a1-c5b10d2efff7 service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:23:54.115536 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a82bc36-4606-401b-90a1-c5b10d2efff7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:23:54.115805 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a82bc36-4606-401b-90a1-c5b10d2efff7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:23:54.117621 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-0a82bc36-4606-401b-90a1-c5b10d2efff7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:23:54.223335 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a82bc36-4606-401b-90a1-c5b10d2efff7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:23:54.224678 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a82bc36-4606-401b-90a1-c5b10d2efff7 service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:23:54.242729 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 575/1144] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:23:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:23:56.437859 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8e8f2e-0402-42ca-b337-15092b5f2898 None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:23:56.438440 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8e8f2e-0402-42ca-b337-15092b5f2898 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:23:56.438440 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8e8f2e-0402-42ca-b337-15092b5f2898 None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:23:56.439770 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 570/1145] 127.0.0.1 () {54 vars in 917 bytes} [Wed Nov 2 07:23:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:23:56.445024 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e076da92-a24f-45d6-9465-21fd37c6bdac None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:23:56.445436 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e076da92-a24f-45d6-9465-21fd37c6bdac None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:23:56.445593 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e076da92-a24f-45d6-9465-21fd37c6bdac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:23:56.513627 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-e076da92-a24f-45d6-9465-21fd37c6bdac None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:23:56.514857 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-e076da92-a24f-45d6-9465-21fd37c6bdac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:23:56.562587 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:23:56.610755 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 576/1146] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Nov 2 07:23:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:23:56.896682 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8e8f2e-0402-42ca-b337-15092b5f2898 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:23:56.898217 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-5e8e8f2e-0402-42ca-b337-15092b5f2898 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:23:57.000189 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8e8f2e-0402-42ca-b337-15092b5f2898 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:23:57.047857 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:23:57.072883 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10de792-d4a7-413d-9392-1b878e90292f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3ec6ecb85f94ef59762877d9ea33ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:23:57.073510 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10de792-d4a7-413d-9392-1b878e90292f service neutron] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:23:57.074325 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10de792-d4a7-413d-9392-1b878e90292f service neutron] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:23:57.074630 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10de792-d4a7-413d-9392-1b878e90292f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:23:57.076220 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-e10de792-d4a7-413d-9392-1b878e90292f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:23:57.185393 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10de792-d4a7-413d-9392-1b878e90292f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:23:57.186768 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10de792-d4a7-413d-9392-1b878e90292f service neutron] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:23:57.205618 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 571/1147] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:23:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:24:06.609159 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=22 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:24:07.203459 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=66 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:24:17.068425 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0120eff1-cac1-4190-b924-20aa29ed01e0 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:24:17.068981 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0120eff1-cac1-4190-b924-20aa29ed01e0 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:24:17.068981 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0120eff1-cac1-4190-b924-20aa29ed01e0 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:24:17.070331 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 577/1148] 10.222.0.18 () {54 vars in 927 bytes} [Wed Nov 2 07:24:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:24:17.076888 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10713f9-86d1-4d87-9ade-894fc7eb1381 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:24:17.077370 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10713f9-86d1-4d87-9ade-894fc7eb1381 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:24:17.077474 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10713f9-86d1-4d87-9ade-894fc7eb1381 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:24:17.141436 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-a10713f9-86d1-4d87-9ade-894fc7eb1381 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:24:17.142778 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-a10713f9-86d1-4d87-9ade-894fc7eb1381 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:24:17.235685 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 572/1149] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Nov 2 07:24:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:24:17.540339 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef67fb5-d2d6-46ae-ae8b-023e5c506b1b None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:24:17.540621 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef67fb5-d2d6-46ae-ae8b-023e5c506b1b None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:24:17.540890 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef67fb5-d2d6-46ae-ae8b-023e5c506b1b None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:24:17.542420 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 578/1150] 10.222.0.18 () {54 vars in 962 bytes} [Wed Nov 2 07:24:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:24:17.547221 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873c4a91-ac12-4aae-a6aa-f8fac65a30ce None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:24:17.547506 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873c4a91-ac12-4aae-a6aa-f8fac65a30ce None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:24:17.547775 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873c4a91-ac12-4aae-a6aa-f8fac65a30ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:24:17.622733 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-873c4a91-ac12-4aae-a6aa-f8fac65a30ce None None] MFA Rules not processed for user `c5353f2af0f44d668e2a269647aa1dc2`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:24:17.623921 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-873c4a91-ac12-4aae-a6aa-f8fac65a30ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:24:17.644614 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:24:17.702104 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 573/1151] 10.222.0.18 () {58 vars in 1059 bytes} [Wed Nov 2 07:24:17 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:24:17.709845 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef67fb5-d2d6-46ae-ae8b-023e5c506b1b None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:24:17.711410 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-8ef67fb5-d2d6-46ae-ae8b-023e5c506b1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:24:17.813458 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef67fb5-d2d6-46ae-ae8b-023e5c506b1b None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:24:17.885285 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad5027a-c56f-40f0-99e5-dc6cf357aabf service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5353f2af0f44d668e2a269647aa1dc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:24:17.885889 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad5027a-c56f-40f0-99e5-dc6cf357aabf service glance] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:24:17.886168 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad5027a-c56f-40f0-99e5-dc6cf357aabf service glance] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:24:17.886470 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad5027a-c56f-40f0-99e5-dc6cf357aabf service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:24:17.888640 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-fad5027a-c56f-40f0-99e5-dc6cf357aabf service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:24:17.942808 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:24:17.998067 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad5027a-c56f-40f0-99e5-dc6cf357aabf service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:24:17.999524 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad5027a-c56f-40f0-99e5-dc6cf357aabf service glance] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:24:18.018152 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 579/1152] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:24:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:24:20.276397 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2b6735-722c-4e9e-aa92-a162cf169893 None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:24:20.276947 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2b6735-722c-4e9e-aa92-a162cf169893 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:24:20.276947 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2b6735-722c-4e9e-aa92-a162cf169893 None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:24:20.278292 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 574/1153] 10.222.0.18 () {54 vars in 927 bytes} [Wed Nov 2 07:24:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:24:20.283889 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9eadaa-ccbb-48c8-9862-5f1c0b64d261 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:24:20.284329 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9eadaa-ccbb-48c8-9862-5f1c0b64d261 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:24:20.284434 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9eadaa-ccbb-48c8-9862-5f1c0b64d261 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:24:20.348735 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-1a9eadaa-ccbb-48c8-9862-5f1c0b64d261 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:24:20.349924 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-1a9eadaa-ccbb-48c8-9862-5f1c0b64d261 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:24:20.445513 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 580/1154] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Nov 2 07:24:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:24:20.725019 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2b6735-722c-4e9e-aa92-a162cf169893 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:24:20.726612 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-1a2b6735-722c-4e9e-aa92-a162cf169893 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:24:20.850493 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2b6735-722c-4e9e-aa92-a162cf169893 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:24:20.901783 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:24:20.944920 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea95d9b-6a1e-4839-989a-e94589ff5dcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:24:20.945518 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea95d9b-6a1e-4839-989a-e94589ff5dcc admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:24:20.945792 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea95d9b-6a1e-4839-989a-e94589ff5dcc admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:24:20.946056 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea95d9b-6a1e-4839-989a-e94589ff5dcc admin admin] PATH_INFO: `/v3/projects` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:24:20.946709 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea95d9b-6a1e-4839-989a-e94589ff5dcc admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:24:20.948437 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: WARNING py.warnings [None req-eea95d9b-6a1e-4839-989a-e94589ff5dcc admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Nov 02 07:24:20.948437 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: warnings.warn(msg) Nov 02 07:24:20.948437 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: Nov 02 07:24:20.949563 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea95d9b-6a1e-4839-989a-e94589ff5dcc admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:24:20.968700 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 575/1155] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Nov 2 07:24:20 2022] GET /identity/v3/projects => generated 2553 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:24:23.145405 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1169ba-234b-42d7-9703-f24d5b55019f None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:24:23.145978 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1169ba-234b-42d7-9703-f24d5b55019f None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:24:23.145978 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1169ba-234b-42d7-9703-f24d5b55019f None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:24:23.147336 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 581/1156] 10.222.0.18 () {54 vars in 927 bytes} [Wed Nov 2 07:24:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:24:23.153148 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f4c5ab-39d4-4839-a245-f02a65b60426 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:24:23.153568 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f4c5ab-39d4-4839-a245-f02a65b60426 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:24:23.153678 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f4c5ab-39d4-4839-a245-f02a65b60426 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:24:23.219803 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-44f4c5ab-39d4-4839-a245-f02a65b60426 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:24:23.221030 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-44f4c5ab-39d4-4839-a245-f02a65b60426 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:24:23.315589 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 576/1157] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Nov 2 07:24:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:24:23.609014 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1169ba-234b-42d7-9703-f24d5b55019f None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:24:23.610545 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-3c1169ba-234b-42d7-9703-f24d5b55019f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:24:23.621195 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:24:23.757271 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1169ba-234b-42d7-9703-f24d5b55019f None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:24:23.836865 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:24:23.870659 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0649048-2dc0-4748-9dd1-a6e85a6c95b9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e48cba3f804cf08b2477f01ae95179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:24:23.871192 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0649048-2dc0-4748-9dd1-a6e85a6c95b9 service nova] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:24:23.871469 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0649048-2dc0-4748-9dd1-a6e85a6c95b9 service nova] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:24:23.871738 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0649048-2dc0-4748-9dd1-a6e85a6c95b9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:24:23.873322 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-e0649048-2dc0-4748-9dd1-a6e85a6c95b9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:24:23.968075 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0649048-2dc0-4748-9dd1-a6e85a6c95b9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:24:23.969640 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0649048-2dc0-4748-9dd1-a6e85a6c95b9 service nova] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:24:23.986855 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 582/1158] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:24:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:24:26.307829 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2507b1-3506-4162-85fe-3c5bd1de60d5 None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:24:26.308410 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2507b1-3506-4162-85fe-3c5bd1de60d5 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:24:26.308410 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2507b1-3506-4162-85fe-3c5bd1de60d5 None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:24:26.309685 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 577/1159] 10.222.0.18 () {54 vars in 927 bytes} [Wed Nov 2 07:24:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:24:26.315063 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d90bc49-ac4d-4076-b322-ce1f38290240 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:24:26.315514 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d90bc49-ac4d-4076-b322-ce1f38290240 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:24:26.315624 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d90bc49-ac4d-4076-b322-ce1f38290240 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:24:26.382966 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-5d90bc49-ac4d-4076-b322-ce1f38290240 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:24:26.384183 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-5d90bc49-ac4d-4076-b322-ce1f38290240 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:24:26.431631 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:24:26.482024 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 583/1160] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Nov 2 07:24:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:24:26.772173 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2507b1-3506-4162-85fe-3c5bd1de60d5 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:24:26.773692 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-eb2507b1-3506-4162-85fe-3c5bd1de60d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:24:26.797071 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:24:26.877030 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2507b1-3506-4162-85fe-3c5bd1de60d5 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:24:26.949261 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed7f652-d5b7-4fdb-8071-dc3fabfc1e3b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3ec6ecb85f94ef59762877d9ea33ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:24:26.949790 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed7f652-d5b7-4fdb-8071-dc3fabfc1e3b service neutron] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:24:26.950069 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed7f652-d5b7-4fdb-8071-dc3fabfc1e3b service neutron] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:24:26.950343 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed7f652-d5b7-4fdb-8071-dc3fabfc1e3b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:24:26.951938 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-fed7f652-d5b7-4fdb-8071-dc3fabfc1e3b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:24:27.026668 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:24:27.062442 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed7f652-d5b7-4fdb-8071-dc3fabfc1e3b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:24:27.063871 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed7f652-d5b7-4fdb-8071-dc3fabfc1e3b service neutron] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:24:27.081608 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 578/1161] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:24:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:24:29.280609 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f17e479-9494-4dc4-a805-b16a3a25ab72 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:24:29.281171 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f17e479-9494-4dc4-a805-b16a3a25ab72 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:24:29.281171 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f17e479-9494-4dc4-a805-b16a3a25ab72 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:24:29.282563 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 584/1162] 10.222.0.18 () {54 vars in 927 bytes} [Wed Nov 2 07:24:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:24:29.287926 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ba58dc-7c84-45d4-a41c-70b474c7ca6a None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:24:29.288365 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ba58dc-7c84-45d4-a41c-70b474c7ca6a None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:24:29.288470 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ba58dc-7c84-45d4-a41c-70b474c7ca6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:24:29.353144 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-66ba58dc-7c84-45d4-a41c-70b474c7ca6a None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:24:29.354497 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-66ba58dc-7c84-45d4-a41c-70b474c7ca6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:24:29.451420 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 579/1163] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Nov 2 07:24:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:24:29.739353 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f17e479-9494-4dc4-a805-b16a3a25ab72 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:24:29.741123 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-4f17e479-9494-4dc4-a805-b16a3a25ab72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:24:29.841046 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f17e479-9494-4dc4-a805-b16a3a25ab72 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:24:29.912517 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:24:29.913983 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6f9fe4-980a-4c7f-aec7-25be53bddb20 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3ec6ecb85f94ef59762877d9ea33ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:24:29.914610 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6f9fe4-980a-4c7f-aec7-25be53bddb20 service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:24:29.915069 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6f9fe4-980a-4c7f-aec7-25be53bddb20 service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:24:29.915656 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6f9fe4-980a-4c7f-aec7-25be53bddb20 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:24:29.917441 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-8e6f9fe4-980a-4c7f-aec7-25be53bddb20 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:24:30.026148 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6f9fe4-980a-4c7f-aec7-25be53bddb20 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:24:30.027597 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6f9fe4-980a-4c7f-aec7-25be53bddb20 service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:24:30.045709 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 585/1164] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:24:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:24:32.377900 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00c67a6-f365-4a58-92d2-86347d425bc0 None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:24:32.378647 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00c67a6-f365-4a58-92d2-86347d425bc0 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:24:32.378647 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00c67a6-f365-4a58-92d2-86347d425bc0 None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:24:32.379830 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 580/1165] 10.222.0.18 () {54 vars in 927 bytes} [Wed Nov 2 07:24:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:24:32.385309 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0a0bd0-45c2-4a9b-b76c-f2060cc2e2fe None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:24:32.385778 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0a0bd0-45c2-4a9b-b76c-f2060cc2e2fe None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:24:32.385885 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0a0bd0-45c2-4a9b-b76c-f2060cc2e2fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:24:32.450668 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-ab0a0bd0-45c2-4a9b-b76c-f2060cc2e2fe None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:24:32.451870 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-ab0a0bd0-45c2-4a9b-b76c-f2060cc2e2fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:24:32.504047 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:24:32.547796 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 586/1166] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Nov 2 07:24:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:24:32.840659 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00c67a6-f365-4a58-92d2-86347d425bc0 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:24:32.842173 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-c00c67a6-f365-4a58-92d2-86347d425bc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:24:32.877648 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:24:32.943718 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00c67a6-f365-4a58-92d2-86347d425bc0 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:24:33.015571 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fa7c8e-b9a0-4432-b1ce-6819f5986372 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3ec6ecb85f94ef59762877d9ea33ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:24:33.016168 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fa7c8e-b9a0-4432-b1ce-6819f5986372 service neutron] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:24:33.016450 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fa7c8e-b9a0-4432-b1ce-6819f5986372 service neutron] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:24:33.016719 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fa7c8e-b9a0-4432-b1ce-6819f5986372 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:24:33.018278 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-d6fa7c8e-b9a0-4432-b1ce-6819f5986372 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:24:33.096446 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:24:33.125248 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fa7c8e-b9a0-4432-b1ce-6819f5986372 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:24:33.126698 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fa7c8e-b9a0-4432-b1ce-6819f5986372 service neutron] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:24:33.144443 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 581/1167] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:24:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:24:35.468087 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1f9d88-ca28-4dad-b0a3-bf2787deca3d None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:24:35.468645 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1f9d88-ca28-4dad-b0a3-bf2787deca3d None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:24:35.468645 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1f9d88-ca28-4dad-b0a3-bf2787deca3d None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:24:35.469942 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 587/1168] 10.222.0.18 () {54 vars in 927 bytes} [Wed Nov 2 07:24:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:24:35.475306 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03aa6c00-6386-4ca4-aa0c-b28bac9d764d None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:24:35.475726 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03aa6c00-6386-4ca4-aa0c-b28bac9d764d None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:24:35.475832 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03aa6c00-6386-4ca4-aa0c-b28bac9d764d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:24:35.544215 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-03aa6c00-6386-4ca4-aa0c-b28bac9d764d None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:24:35.545423 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-03aa6c00-6386-4ca4-aa0c-b28bac9d764d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:24:35.643204 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 582/1169] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Nov 2 07:24:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:24:35.929532 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1f9d88-ca28-4dad-b0a3-bf2787deca3d None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:24:35.931427 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-4d1f9d88-ca28-4dad-b0a3-bf2787deca3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:24:36.036910 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1f9d88-ca28-4dad-b0a3-bf2787deca3d None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:24:36.111510 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a188f3-c2b6-43fe-add2-ded942018f17 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3ec6ecb85f94ef59762877d9ea33ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:24:36.112119 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a188f3-c2b6-43fe-add2-ded942018f17 service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:24:36.112396 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a188f3-c2b6-43fe-add2-ded942018f17 service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:24:36.112661 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a188f3-c2b6-43fe-add2-ded942018f17 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:24:36.114252 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-c7a188f3-c2b6-43fe-add2-ded942018f17 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:24:36.123123 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:24:36.221906 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a188f3-c2b6-43fe-add2-ded942018f17 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:24:36.223402 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a188f3-c2b6-43fe-add2-ded942018f17 service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:24:36.241588 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 588/1170] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:24:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:24:45.641296 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=82 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:24:46.238859 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=50 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:25:49.000243 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301038ea-e55b-47cc-a2bd-ef38a1a21270 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:25:49.000826 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301038ea-e55b-47cc-a2bd-ef38a1a21270 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:25:49.000826 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301038ea-e55b-47cc-a2bd-ef38a1a21270 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:25:49.075204 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-301038ea-e55b-47cc-a2bd-ef38a1a21270 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:25:49.076386 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-301038ea-e55b-47cc-a2bd-ef38a1a21270 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:25:49.173242 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 583/1171] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:25:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:25:49.181254 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb8df2b-29d4-4f54-be70-69b21428e29e None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:25:49.181800 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb8df2b-29d4-4f54-be70-69b21428e29e None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:25:49.181800 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb8df2b-29d4-4f54-be70-69b21428e29e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:25:49.255408 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-1cb8df2b-29d4-4f54-be70-69b21428e29e None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:25:49.256516 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-1cb8df2b-29d4-4f54-be70-69b21428e29e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:25:49.351379 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 589/1172] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:25:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:25:49.356010 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301038ea-e55b-47cc-a2bd-ef38a1a21270 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:25:49.357203 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-301038ea-e55b-47cc-a2bd-ef38a1a21270 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:25:49.368822 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301038ea-e55b-47cc-a2bd-ef38a1a21270 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:25:49.430182 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:25:49.464299 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be768196-7a94-48ea-abab-32e909cf8be7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:25:49.464899 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be768196-7a94-48ea-abab-32e909cf8be7 admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:25:49.465174 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be768196-7a94-48ea-abab-32e909cf8be7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:25:49.465442 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be768196-7a94-48ea-abab-32e909cf8be7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:25:49.466078 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be768196-7a94-48ea-abab-32e909cf8be7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:25:49.466399 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be768196-7a94-48ea-abab-32e909cf8be7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:25:49.467420 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: WARNING py.warnings [None req-be768196-7a94-48ea-abab-32e909cf8be7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Nov 02 07:25:49.467420 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: warnings.warn(msg) Nov 02 07:25:49.467420 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: Nov 02 07:25:49.468293 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be768196-7a94-48ea-abab-32e909cf8be7 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:25:49.480208 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 584/1173] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Nov 2 07:25:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:25:49.490910 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb8df2b-29d4-4f54-be70-69b21428e29e None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:25:49.500193 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb8df2b-29d4-4f54-be70-69b21428e29e None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:25:49.561030 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:25:49.593938 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c45ab37-77d5-435e-8457-b4d247f0a612 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:25:49.594585 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c45ab37-77d5-435e-8457-b4d247f0a612 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:25:49.594861 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c45ab37-77d5-435e-8457-b4d247f0a612 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:25:49.595128 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c45ab37-77d5-435e-8457-b4d247f0a612 admin admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:25:49.595755 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c45ab37-77d5-435e-8457-b4d247f0a612 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:25:49.596761 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: WARNING py.warnings [None req-2c45ab37-77d5-435e-8457-b4d247f0a612 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Nov 02 07:25:49.596761 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: warnings.warn(msg) Nov 02 07:25:49.596761 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: Nov 02 07:25:49.597943 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c45ab37-77d5-435e-8457-b4d247f0a612 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:25:49.653563 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 590/1174] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Nov 2 07:25:49 2022] POST /identity/v3/projects => generated 377 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:25:49.657648 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be768196-7a94-48ea-abab-32e909cf8be7 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:25:49.659201 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-be768196-7a94-48ea-abab-32e909cf8be7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:25:49.692997 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be768196-7a94-48ea-abab-32e909cf8be7 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:25:49.784471 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23db659d-3c97-47fb-8a8e-37c2d0f4144f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:25:49.785074 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23db659d-3c97-47fb-8a8e-37c2d0f4144f admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:25:49.785352 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23db659d-3c97-47fb-8a8e-37c2d0f4144f admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:25:49.785624 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23db659d-3c97-47fb-8a8e-37c2d0f4144f admin admin] PATH_INFO: `/v3/users` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:25:49.786244 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23db659d-3c97-47fb-8a8e-37c2d0f4144f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:25:49.787978 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: WARNING py.warnings [None req-23db659d-3c97-47fb-8a8e-37c2d0f4144f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Nov 02 07:25:49.787978 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: warnings.warn(msg) Nov 02 07:25:49.787978 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: Nov 02 07:25:49.789180 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23db659d-3c97-47fb-8a8e-37c2d0f4144f admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:25:49.830676 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 585/1175] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Nov 2 07:25:49 2022] POST /identity/v3/users => generated 289 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:25:49.835456 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c45ab37-77d5-435e-8457-b4d247f0a612 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:25:49.837002 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-2c45ab37-77d5-435e-8457-b4d247f0a612 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:25:49.898887 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c45ab37-77d5-435e-8457-b4d247f0a612 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:25:49.981437 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:25:49.992925 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef031913-91c9-4a77-93b0-9d6807b7469a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:25:49.993529 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef031913-91c9-4a77-93b0-9d6807b7469a admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:25:49.993808 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef031913-91c9-4a77-93b0-9d6807b7469a admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:25:49.994078 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef031913-91c9-4a77-93b0-9d6807b7469a admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:25:49.994755 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef031913-91c9-4a77-93b0-9d6807b7469a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:25:49.996347 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef031913-91c9-4a77-93b0-9d6807b7469a admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:25:50.008378 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 591/1176] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:25:49 2022] GET /identity/v3/roles => generated 1420 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:25:50.012576 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23db659d-3c97-47fb-8a8e-37c2d0f4144f admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:25:50.014089 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-23db659d-3c97-47fb-8a8e-37c2d0f4144f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:25:50.025878 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23db659d-3c97-47fb-8a8e-37c2d0f4144f admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:25:50.050706 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:25:50.122546 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72e1e6f-4189-4b10-b6d1-a61d0e685109 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:25:50.123068 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72e1e6f-4189-4b10-b6d1-a61d0e685109 admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:25:50.123351 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72e1e6f-4189-4b10-b6d1-a61d0e685109 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:25:50.123620 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72e1e6f-4189-4b10-b6d1-a61d0e685109 admin admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:25:50.124251 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72e1e6f-4189-4b10-b6d1-a61d0e685109 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:25:50.125833 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72e1e6f-4189-4b10-b6d1-a61d0e685109 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:25:50.137946 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 586/1177] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:25:50 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:25:50.142133 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef031913-91c9-4a77-93b0-9d6807b7469a admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:25:50.143666 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-ef031913-91c9-4a77-93b0-9d6807b7469a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:25:50.177296 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef031913-91c9-4a77-93b0-9d6807b7469a admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:25:50.272634 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c2fdb0-271b-46ff-9ed7-3ac0653a7f48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:25:50.273324 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c2fdb0-271b-46ff-9ed7-3ac0653a7f48 admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:25:50.273538 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c2fdb0-271b-46ff-9ed7-3ac0653a7f48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:25:50.273805 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c2fdb0-271b-46ff-9ed7-3ac0653a7f48 admin admin] PATH_INFO: `/v3/projects/edc15b8fb9764dc2a0067e5e25e14956/users/afd63a24b44b4742bb2415d17ab3036e/roles/b494fedd00f24f0b84a529e8672a156a` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:25:50.296290 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c2fdb0-271b-46ff-9ed7-3ac0653a7f48 admin admin] RBAC: Authorizing `identity:create_grant(project_id=edc15b8fb9764dc2a0067e5e25e14956, user_id=afd63a24b44b4742bb2415d17ab3036e, role_id=b494fedd00f24f0b84a529e8672a156a)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:25:50.299016 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c2fdb0-271b-46ff-9ed7-3ac0653a7f48 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:25:50.323375 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 592/1178] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Nov 2 07:25:50 2022] PUT /identity/v3/projects/edc15b8fb9764dc2a0067e5e25e14956/users/afd63a24b44b4742bb2415d17ab3036e/roles/b494fedd00f24f0b84a529e8672a156a => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:25:50.334460 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d31e73-e359-4623-bb36-45fd78aa0727 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:25:50.334755 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d31e73-e359-4623-bb36-45fd78aa0727 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:25:50.335037 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d31e73-e359-4623-bb36-45fd78aa0727 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:25:50.392564 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-a1d31e73-e359-4623-bb36-45fd78aa0727 None None] MFA Rules not processed for user `afd63a24b44b4742bb2415d17ab3036e`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:25:50.393878 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-a1d31e73-e359-4623-bb36-45fd78aa0727 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:25:50.469365 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 587/1179] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:25:50 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:25:50.482904 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6506d55e-5efc-40ca-b19f-56871027a903 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:25:50.483190 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6506d55e-5efc-40ca-b19f-56871027a903 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:25:50.483462 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6506d55e-5efc-40ca-b19f-56871027a903 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:25:50.484780 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 593/1180] 10.222.0.18 () {54 vars in 963 bytes} [Wed Nov 2 07:25:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:25:50.491138 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e719eb9-35b2-40da-98de-0ac26739005d None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:25:50.491426 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e719eb9-35b2-40da-98de-0ac26739005d None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:25:50.491706 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e719eb9-35b2-40da-98de-0ac26739005d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:25:50.523555 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:25:50.570473 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-0e719eb9-35b2-40da-98de-0ac26739005d None None] MFA Rules not processed for user `43bb326fb82e4d2c83fb2eaf91860dc8`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:25:50.571679 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-0e719eb9-35b2-40da-98de-0ac26739005d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:25:50.651105 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 588/1181] 10.222.0.18 () {58 vars in 1060 bytes} [Wed Nov 2 07:25:50 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:25:50.658904 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6506d55e-5efc-40ca-b19f-56871027a903 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:25:50.660471 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-6506d55e-5efc-40ca-b19f-56871027a903 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:25:50.710864 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:25:50.764198 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6506d55e-5efc-40ca-b19f-56871027a903 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:25:50.837401 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01557f24-f746-4ddb-bf38-d88559494c27 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43bb326fb82e4d2c83fb2eaf91860dc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:25:50.838001 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01557f24-f746-4ddb-bf38-d88559494c27 service cinder] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:25:50.838278 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01557f24-f746-4ddb-bf38-d88559494c27 service cinder] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:25:50.838579 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01557f24-f746-4ddb-bf38-d88559494c27 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:25:50.840157 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-01557f24-f746-4ddb-bf38-d88559494c27 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:25:50.941448 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01557f24-f746-4ddb-bf38-d88559494c27 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:25:50.943371 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01557f24-f746-4ddb-bf38-d88559494c27 service cinder] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:25:50.946830 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:25:50.962240 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 594/1182] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:25:50 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:25:51.008157 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89445049-1e8a-43a2-9e2e-c5593b0da05e None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:25:51.008445 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89445049-1e8a-43a2-9e2e-c5593b0da05e None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:25:51.008717 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89445049-1e8a-43a2-9e2e-c5593b0da05e None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:25:51.009975 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 589/1183] 10.222.0.18 () {52 vars in 830 bytes} [Wed Nov 2 07:25:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:25:51.013562 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01557f24-f746-4ddb-bf38-d88559494c27 service cinder] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:25:51.014789 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-01557f24-f746-4ddb-bf38-d88559494c27 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:25:51.111309 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01557f24-f746-4ddb-bf38-d88559494c27 service cinder] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:25:51.201940 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cc6331-03e6-4613-b5fe-dcd1fc71716a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:25:51.202513 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cc6331-03e6-4613-b5fe-dcd1fc71716a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:25:51.202787 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cc6331-03e6-4613-b5fe-dcd1fc71716a admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:25:51.203052 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cc6331-03e6-4613-b5fe-dcd1fc71716a admin admin] PATH_INFO: `/v3/users/afd63a24b44b4742bb2415d17ab3036e` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:25:51.216071 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cc6331-03e6-4613-b5fe-dcd1fc71716a admin admin] RBAC: Authorizing `identity:delete_user(user_id=afd63a24b44b4742bb2415d17ab3036e)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:25:51.217072 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: WARNING py.warnings [None req-71cc6331-03e6-4613-b5fe-dcd1fc71716a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Nov 02 07:25:51.217072 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: warnings.warn(msg) Nov 02 07:25:51.217072 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: Nov 02 07:25:51.218271 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cc6331-03e6-4613-b5fe-dcd1fc71716a admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:25:51.227859 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:25:51.288324 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-71cc6331-03e6-4613-b5fe-dcd1fc71716a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'afd63a24b44b4742bb2415d17ab3036e'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:25:51.297758 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-71cc6331-03e6-4613-b5fe-dcd1fc71716a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'afd63a24b44b4742bb2415d17ab3036e'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:25:51.298187 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-71cc6331-03e6-4613-b5fe-dcd1fc71716a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'afd63a24b44b4742bb2415d17ab3036e'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:25:51.312956 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-71cc6331-03e6-4613-b5fe-dcd1fc71716a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'afd63a24b44b4742bb2415d17ab3036e'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:25:51.313488 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-71cc6331-03e6-4613-b5fe-dcd1fc71716a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'afd63a24b44b4742bb2415d17ab3036e'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:25:51.341336 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 595/1184] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Nov 2 07:25:51 2022] DELETE /identity/v3/users/afd63a24b44b4742bb2415d17ab3036e => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:25:51.345685 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8568b9a-ef31-4100-9799-2cfd3d1b8488 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:25:51.346191 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8568b9a-ef31-4100-9799-2cfd3d1b8488 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:25:51.346290 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8568b9a-ef31-4100-9799-2cfd3d1b8488 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:25:51.406112 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-a8568b9a-ef31-4100-9799-2cfd3d1b8488 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:25:51.407369 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-a8568b9a-ef31-4100-9799-2cfd3d1b8488 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:25:51.482154 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:25:51.495850 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 590/1185] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:25:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:25:51.506166 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cc6331-03e6-4613-b5fe-dcd1fc71716a admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:25:51.507450 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-71cc6331-03e6-4613-b5fe-dcd1fc71716a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:25:51.607735 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cc6331-03e6-4613-b5fe-dcd1fc71716a admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:25:51.662613 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:25:51.684104 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6eab0d1-04d1-4f88-8537-30020b417f92 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3ec6ecb85f94ef59762877d9ea33ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:25:51.684717 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6eab0d1-04d1-4f88-8537-30020b417f92 service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:25:51.684990 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6eab0d1-04d1-4f88-8537-30020b417f92 service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:25:51.685255 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6eab0d1-04d1-4f88-8537-30020b417f92 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:25:51.686850 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-b6eab0d1-04d1-4f88-8537-30020b417f92 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:25:51.795285 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6eab0d1-04d1-4f88-8537-30020b417f92 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:25:51.796726 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6eab0d1-04d1-4f88-8537-30020b417f92 service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:25:51.814262 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 596/1186] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:25:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:25:52.166829 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8568b9a-ef31-4100-9799-2cfd3d1b8488 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:25:52.168053 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-a8568b9a-ef31-4100-9799-2cfd3d1b8488 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:25:52.180040 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8568b9a-ef31-4100-9799-2cfd3d1b8488 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:25:52.276829 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5711f507-d180-4422-80fc-ec6e700214da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:25:52.277454 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5711f507-d180-4422-80fc-ec6e700214da admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:25:52.277727 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5711f507-d180-4422-80fc-ec6e700214da admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:25:52.277992 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5711f507-d180-4422-80fc-ec6e700214da admin admin] PATH_INFO: `/v3/projects/edc15b8fb9764dc2a0067e5e25e14956` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:25:52.288433 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5711f507-d180-4422-80fc-ec6e700214da admin admin] RBAC: Authorizing `identity:delete_project(project_id=edc15b8fb9764dc2a0067e5e25e14956)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:25:52.289617 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: WARNING py.warnings [None req-5711f507-d180-4422-80fc-ec6e700214da admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Nov 02 07:25:52.289617 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: warnings.warn(msg) Nov 02 07:25:52.289617 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: Nov 02 07:25:52.290839 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5711f507-d180-4422-80fc-ec6e700214da admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:25:52.346769 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-5711f507-d180-4422-80fc-ec6e700214da admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'edc15b8fb9764dc2a0067e5e25e14956', 'request_id': 'req-5711f507-d180-4422-80fc-ec6e700214da'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:25:52.360089 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-5711f507-d180-4422-80fc-ec6e700214da admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'edc15b8fb9764dc2a0067e5e25e14956', 'request_id': 'req-5711f507-d180-4422-80fc-ec6e700214da'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:25:52.370310 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-5711f507-d180-4422-80fc-ec6e700214da admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'edc15b8fb9764dc2a0067e5e25e14956', 'request_id': 'req-5711f507-d180-4422-80fc-ec6e700214da'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:25:52.395160 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-5711f507-d180-4422-80fc-ec6e700214da admin admin] The token cache is being invalidate because project edc15b8fb9764dc2a0067e5e25e14956 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81501) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:25:52.395443 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-5711f507-d180-4422-80fc-ec6e700214da admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:25:52.395867 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-5711f507-d180-4422-80fc-ec6e700214da admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:25:52.397259 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 591/1187] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Nov 2 07:25:52 2022] DELETE /identity/v3/projects/edc15b8fb9764dc2a0067e5e25e14956 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:26:01.812066 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=62 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:26:02.392846 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:27:19.898408 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e898f7a3-be70-46bb-824e-64cd3d212fd2 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:19.898965 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e898f7a3-be70-46bb-824e-64cd3d212fd2 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:19.899080 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e898f7a3-be70-46bb-824e-64cd3d212fd2 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:19.900996 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 597/1188] 127.0.0.1 () {54 vars in 901 bytes} [Wed Nov 2 07:27:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:27:19.905754 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763a6955-008e-4952-aaef-81a310bb6b3a None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:19.906207 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763a6955-008e-4952-aaef-81a310bb6b3a None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:19.906313 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763a6955-008e-4952-aaef-81a310bb6b3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:19.986598 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-763a6955-008e-4952-aaef-81a310bb6b3a None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:27:19.987789 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-763a6955-008e-4952-aaef-81a310bb6b3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:27:20.089675 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 592/1189] 127.0.0.1 () {58 vars in 998 bytes} [Wed Nov 2 07:27:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:27:20.251103 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e898f7a3-be70-46bb-824e-64cd3d212fd2 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:20.253173 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-e898f7a3-be70-46bb-824e-64cd3d212fd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:27:20.370044 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e898f7a3-be70-46bb-824e-64cd3d212fd2 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:20.375410 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0a3450-507b-4774-9423-10aa482a762c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e48cba3f804cf08b2477f01ae95179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:20.376007 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0a3450-507b-4774-9423-10aa482a762c service nova] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:20.376284 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0a3450-507b-4774-9423-10aa482a762c service nova] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:20.376550 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0a3450-507b-4774-9423-10aa482a762c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:20.385265 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0a3450-507b-4774-9423-10aa482a762c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:20.386873 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0a3450-507b-4774-9423-10aa482a762c service nova] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:20.395947 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 598/1190] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:27:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:27:21.465520 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e568d98-4cda-4890-aeb4-a76363ffc6af None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:21.465976 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e568d98-4cda-4890-aeb4-a76363ffc6af None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:21.466167 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e568d98-4cda-4890-aeb4-a76363ffc6af None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:21.467433 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 593/1191] 127.0.0.1 () {54 vars in 901 bytes} [Wed Nov 2 07:27:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:27:21.472147 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342ffea2-bd51-4abd-9627-e6d982c2bdef None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:21.472554 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342ffea2-bd51-4abd-9627-e6d982c2bdef None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:21.472693 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342ffea2-bd51-4abd-9627-e6d982c2bdef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:21.492870 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-342ffea2-bd51-4abd-9627-e6d982c2bdef None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:27:21.494578 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-342ffea2-bd51-4abd-9627-e6d982c2bdef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:27:21.501987 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 599/1192] 127.0.0.1 () {58 vars in 998 bytes} [Wed Nov 2 07:27:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:27:21.621275 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93723f62-8635-4ef5-aaf3-73026bb48d5f None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:21.621567 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93723f62-8635-4ef5-aaf3-73026bb48d5f None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:21.621847 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93723f62-8635-4ef5-aaf3-73026bb48d5f None None] PATH_INFO: `/v3` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:21.623163 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 594/1193] 10.222.0.47 () {54 vars in 925 bytes} [Wed Nov 2 07:27:21 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:27:21.629670 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d1c032-e703-4bad-ae94-292c9031f8a9 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:21.629955 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d1c032-e703-4bad-ae94-292c9031f8a9 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:21.630226 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d1c032-e703-4bad-ae94-292c9031f8a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:21.655581 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93723f62-8635-4ef5-aaf3-73026bb48d5f None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:21.670941 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93723f62-8635-4ef5-aaf3-73026bb48d5f None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:21.676329 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b919778-80f7-4c74-877c-b314756f92cf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e48cba3f804cf08b2477f01ae95179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:21.676925 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b919778-80f7-4c74-877c-b314756f92cf service nova] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:21.677203 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b919778-80f7-4c74-877c-b314756f92cf service nova] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:21.677481 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b919778-80f7-4c74-877c-b314756f92cf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:21.686732 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b919778-80f7-4c74-877c-b314756f92cf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:21.688316 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b919778-80f7-4c74-877c-b314756f92cf service nova] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:21.691732 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-07d1c032-e703-4bad-ae94-292c9031f8a9 None None] MFA Rules not processed for user `4f2fc6a156c54d0c9d325901d0f7f6fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:27:21.692917 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-07d1c032-e703-4bad-ae94-292c9031f8a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:27:21.698599 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 595/1194] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:27:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 07:27:21.748621 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:27:21.769685 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 600/1195] 10.222.0.47 () {58 vars in 1015 bytes} [Wed Nov 2 07:27:21 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:27:30.299283 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c82c928-4296-4982-bcec-aa7f9bc79463 None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:30.299790 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c82c928-4296-4982-bcec-aa7f9bc79463 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:30.299908 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c82c928-4296-4982-bcec-aa7f9bc79463 None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:30.301140 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 596/1196] 10.222.0.18 () {54 vars in 961 bytes} [Wed Nov 2 07:27:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:27:30.306160 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1036eb-c770-48ac-86f0-75673f1b5ee1 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:30.307223 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1036eb-c770-48ac-86f0-75673f1b5ee1 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:30.307357 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1036eb-c770-48ac-86f0-75673f1b5ee1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:30.327038 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-1c1036eb-c770-48ac-86f0-75673f1b5ee1 None None] MFA Rules not processed for user `4f2fc6a156c54d0c9d325901d0f7f6fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:27:30.328216 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-1c1036eb-c770-48ac-86f0-75673f1b5ee1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:27:30.336546 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 601/1197] 10.222.0.18 () {58 vars in 1058 bytes} [Wed Nov 2 07:27:30 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:27:30.343923 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c82c928-4296-4982-bcec-aa7f9bc79463 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:30.356974 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c82c928-4296-4982-bcec-aa7f9bc79463 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:30.362220 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce3a4bc-3b5e-4f17-8db1-03113cecf0e8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2fc6a156c54d0c9d325901d0f7f6fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:30.362835 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce3a4bc-3b5e-4f17-8db1-03113cecf0e8 service placement] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:30.363110 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce3a4bc-3b5e-4f17-8db1-03113cecf0e8 service placement] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:30.363376 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce3a4bc-3b5e-4f17-8db1-03113cecf0e8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:30.372404 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce3a4bc-3b5e-4f17-8db1-03113cecf0e8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:30.373957 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce3a4bc-3b5e-4f17-8db1-03113cecf0e8 service placement] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:30.383321 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 597/1198] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:27:30 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 07:27:33.160246 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc19149e-8e11-4b52-895c-4b08bcb20dba None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:33.160872 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc19149e-8e11-4b52-895c-4b08bcb20dba None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:33.160872 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc19149e-8e11-4b52-895c-4b08bcb20dba None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:33.162152 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 602/1199] 127.0.0.1 () {54 vars in 901 bytes} [Wed Nov 2 07:27:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:27:33.167225 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45030af7-3603-4123-b2d1-8911315ecbb7 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:33.167650 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45030af7-3603-4123-b2d1-8911315ecbb7 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:33.167753 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45030af7-3603-4123-b2d1-8911315ecbb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:33.187446 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-45030af7-3603-4123-b2d1-8911315ecbb7 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:27:33.188624 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-45030af7-3603-4123-b2d1-8911315ecbb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:27:33.196246 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 598/1200] 127.0.0.1 () {58 vars in 998 bytes} [Wed Nov 2 07:27:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:27:33.351025 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc19149e-8e11-4b52-895c-4b08bcb20dba None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:33.365804 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc19149e-8e11-4b52-895c-4b08bcb20dba None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:33.372005 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1281112-52f5-4122-94ca-e7c04084ee04 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e48cba3f804cf08b2477f01ae95179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:33.372600 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1281112-52f5-4122-94ca-e7c04084ee04 service nova] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:33.372872 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1281112-52f5-4122-94ca-e7c04084ee04 service nova] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:33.373136 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1281112-52f5-4122-94ca-e7c04084ee04 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:33.380996 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1281112-52f5-4122-94ca-e7c04084ee04 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:33.382573 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1281112-52f5-4122-94ca-e7c04084ee04 service nova] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:33.392333 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 603/1201] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:27:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:27:34.445596 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f719cc3c-2e25-45a6-b743-e9a860dcce77 None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:34.446211 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f719cc3c-2e25-45a6-b743-e9a860dcce77 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:34.446211 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f719cc3c-2e25-45a6-b743-e9a860dcce77 None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:34.448030 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 599/1202] 127.0.0.1 () {54 vars in 901 bytes} [Wed Nov 2 07:27:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:27:34.452707 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8a0045-290d-4e38-99fb-ea4af562ba05 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:34.453151 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8a0045-290d-4e38-99fb-ea4af562ba05 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:34.453246 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8a0045-290d-4e38-99fb-ea4af562ba05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:34.473830 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-cc8a0045-290d-4e38-99fb-ea4af562ba05 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:27:34.475055 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-cc8a0045-290d-4e38-99fb-ea4af562ba05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:27:34.482800 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 604/1203] 127.0.0.1 () {58 vars in 998 bytes} [Wed Nov 2 07:27:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:27:34.632971 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f719cc3c-2e25-45a6-b743-e9a860dcce77 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:34.648062 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f719cc3c-2e25-45a6-b743-e9a860dcce77 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:34.654102 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f149ad3-6a81-4f79-88ce-d346e0efb085 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e48cba3f804cf08b2477f01ae95179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:34.654730 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f149ad3-6a81-4f79-88ce-d346e0efb085 service nova] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:34.655013 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f149ad3-6a81-4f79-88ce-d346e0efb085 service nova] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:34.655282 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f149ad3-6a81-4f79-88ce-d346e0efb085 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:34.663438 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f149ad3-6a81-4f79-88ce-d346e0efb085 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:34.665012 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f149ad3-6a81-4f79-88ce-d346e0efb085 service nova] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:34.674900 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 600/1204] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:27:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:27:38.738263 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765fc5f3-af2f-41e5-923b-d8fd945a30db None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:38.738849 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765fc5f3-af2f-41e5-923b-d8fd945a30db None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:38.738849 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765fc5f3-af2f-41e5-923b-d8fd945a30db None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:38.740468 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 605/1205] 127.0.0.1 () {54 vars in 917 bytes} [Wed Nov 2 07:27:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:27:38.746510 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316b33e5-14b4-4681-8b3d-0bf3be5cec85 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:38.746959 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316b33e5-14b4-4681-8b3d-0bf3be5cec85 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:38.747069 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316b33e5-14b4-4681-8b3d-0bf3be5cec85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:38.766012 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-316b33e5-14b4-4681-8b3d-0bf3be5cec85 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:27:38.767233 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-316b33e5-14b4-4681-8b3d-0bf3be5cec85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:27:38.774899 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 601/1206] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Nov 2 07:27:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:27:39.066891 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765fc5f3-af2f-41e5-923b-d8fd945a30db None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:39.068692 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-765fc5f3-af2f-41e5-923b-d8fd945a30db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:27:39.147867 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765fc5f3-af2f-41e5-923b-d8fd945a30db None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:39.151401 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416e8a13-434f-4cf4-9534-bab1e301d990 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3ec6ecb85f94ef59762877d9ea33ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:39.151995 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416e8a13-434f-4cf4-9534-bab1e301d990 service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:39.152278 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416e8a13-434f-4cf4-9534-bab1e301d990 service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:39.152544 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416e8a13-434f-4cf4-9534-bab1e301d990 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:39.161605 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416e8a13-434f-4cf4-9534-bab1e301d990 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:39.163050 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416e8a13-434f-4cf4-9534-bab1e301d990 service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:39.172885 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 606/1207] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:27:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:27:41.310760 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ca4a58-ea84-448f-ba6a-f299b11ff0e3 None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:41.311348 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ca4a58-ea84-448f-ba6a-f299b11ff0e3 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:41.311348 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ca4a58-ea84-448f-ba6a-f299b11ff0e3 None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:41.312628 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 602/1208] 127.0.0.1 () {54 vars in 917 bytes} [Wed Nov 2 07:27:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:27:41.317764 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71d3dda-0df0-4d74-8fdf-7e6e1e2cd3b8 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:41.318151 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71d3dda-0df0-4d74-8fdf-7e6e1e2cd3b8 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:41.318301 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71d3dda-0df0-4d74-8fdf-7e6e1e2cd3b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:41.338261 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-a71d3dda-0df0-4d74-8fdf-7e6e1e2cd3b8 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:27:41.340131 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-a71d3dda-0df0-4d74-8fdf-7e6e1e2cd3b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:27:41.347907 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 607/1209] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Nov 2 07:27:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:27:41.646854 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ca4a58-ea84-448f-ba6a-f299b11ff0e3 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:41.658292 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ca4a58-ea84-448f-ba6a-f299b11ff0e3 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:41.661693 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8341087e-c0c1-4e7d-a06f-a3181730615e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3ec6ecb85f94ef59762877d9ea33ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:41.662280 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8341087e-c0c1-4e7d-a06f-a3181730615e service neutron] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:41.662590 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8341087e-c0c1-4e7d-a06f-a3181730615e service neutron] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:41.662857 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8341087e-c0c1-4e7d-a06f-a3181730615e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:41.672507 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8341087e-c0c1-4e7d-a06f-a3181730615e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:41.673949 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8341087e-c0c1-4e7d-a06f-a3181730615e service neutron] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:41.683370 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 603/1210] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:27:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:27:45.342626 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10c63e7-5033-4b30-ba52-94491bac8572 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:45.343197 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10c63e7-5033-4b30-ba52-94491bac8572 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:45.343197 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10c63e7-5033-4b30-ba52-94491bac8572 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:45.344475 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 608/1211] 127.0.0.1 () {54 vars in 917 bytes} [Wed Nov 2 07:27:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:27:45.349635 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f178027-9658-45a8-aa46-877ba116489a None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:45.350024 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f178027-9658-45a8-aa46-877ba116489a None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:45.350283 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f178027-9658-45a8-aa46-877ba116489a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:45.358131 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:27:45.371947 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-2f178027-9658-45a8-aa46-877ba116489a None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:27:45.373153 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-2f178027-9658-45a8-aa46-877ba116489a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:27:45.380850 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 604/1212] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Nov 2 07:27:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:27:45.681335 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10c63e7-5033-4b30-ba52-94491bac8572 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:45.683639 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-c10c63e7-5033-4b30-ba52-94491bac8572 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:27:45.759103 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10c63e7-5033-4b30-ba52-94491bac8572 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:45.763291 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fb8694-c286-4f98-9a0a-15e5fcdfd3a6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43bb326fb82e4d2c83fb2eaf91860dc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:45.763883 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fb8694-c286-4f98-9a0a-15e5fcdfd3a6 service cinder] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:45.764157 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fb8694-c286-4f98-9a0a-15e5fcdfd3a6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:45.764444 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fb8694-c286-4f98-9a0a-15e5fcdfd3a6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:45.772993 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fb8694-c286-4f98-9a0a-15e5fcdfd3a6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:45.774445 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fb8694-c286-4f98-9a0a-15e5fcdfd3a6 service cinder] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:45.784094 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 609/1213] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:27:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:27:47.928120 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0137efe8-9c7e-4bbb-a249-761c44ad16d2 None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:47.928698 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0137efe8-9c7e-4bbb-a249-761c44ad16d2 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:47.928698 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0137efe8-9c7e-4bbb-a249-761c44ad16d2 None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:47.929987 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 605/1214] 127.0.0.1 () {54 vars in 917 bytes} [Wed Nov 2 07:27:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:27:47.935327 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca26700-f866-4914-ad1b-0057c47e2359 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:47.935735 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca26700-f866-4914-ad1b-0057c47e2359 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:47.935865 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca26700-f866-4914-ad1b-0057c47e2359 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:47.955457 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-dca26700-f866-4914-ad1b-0057c47e2359 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:27:47.956622 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-dca26700-f866-4914-ad1b-0057c47e2359 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:27:47.964816 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 610/1215] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Nov 2 07:27:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:27:48.275943 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0137efe8-9c7e-4bbb-a249-761c44ad16d2 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:48.277598 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-0137efe8-9c7e-4bbb-a249-761c44ad16d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:27:48.290314 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0137efe8-9c7e-4bbb-a249-761c44ad16d2 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:48.293728 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7987e5b6-68d5-419b-854e-68f6ffcfc230 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43bb326fb82e4d2c83fb2eaf91860dc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:48.294305 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7987e5b6-68d5-419b-854e-68f6ffcfc230 service cinder] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:48.294603 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7987e5b6-68d5-419b-854e-68f6ffcfc230 service cinder] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:48.294880 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7987e5b6-68d5-419b-854e-68f6ffcfc230 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:48.303629 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7987e5b6-68d5-419b-854e-68f6ffcfc230 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:48.305106 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7987e5b6-68d5-419b-854e-68f6ffcfc230 service cinder] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:48.315916 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 606/1216] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:27:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:27:57.950628 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=92 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:27:58.311674 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=8 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:29:10.174794 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e4d8db-135e-417d-878a-554baf1e88d5 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:10.175889 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e4d8db-135e-417d-878a-554baf1e88d5 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:10.178782 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e4d8db-135e-417d-878a-554baf1e88d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:10.199867 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-44e4d8db-135e-417d-878a-554baf1e88d5 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:29:10.201098 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-44e4d8db-135e-417d-878a-554baf1e88d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:10.214004 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 611/1217] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:29:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:29:10.225608 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ea77a1-0e63-4439-87a3-72d2e4962037 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:10.226225 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ea77a1-0e63-4439-87a3-72d2e4962037 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:10.226225 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ea77a1-0e63-4439-87a3-72d2e4962037 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:10.255769 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-33ea77a1-0e63-4439-87a3-72d2e4962037 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:29:10.256988 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-33ea77a1-0e63-4439-87a3-72d2e4962037 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:10.271100 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 607/1218] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:29:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:29:10.275831 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e4d8db-135e-417d-878a-554baf1e88d5 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:10.286196 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e4d8db-135e-417d-878a-554baf1e88d5 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:10.295279 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25494c58-ebcc-4cdb-8005-08768a480847 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:10.296195 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25494c58-ebcc-4cdb-8005-08768a480847 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:10.296709 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25494c58-ebcc-4cdb-8005-08768a480847 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:10.297335 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25494c58-ebcc-4cdb-8005-08768a480847 admin admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:10.298331 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25494c58-ebcc-4cdb-8005-08768a480847 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:29:10.298868 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25494c58-ebcc-4cdb-8005-08768a480847 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:10.300104 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: WARNING py.warnings [None req-25494c58-ebcc-4cdb-8005-08768a480847 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Nov 02 07:29:10.300104 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: warnings.warn(msg) Nov 02 07:29:10.300104 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: Nov 02 07:29:10.301418 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25494c58-ebcc-4cdb-8005-08768a480847 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:10.319887 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92be89e-02de-4bb8-a697-9baaf95cd860 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:10.320249 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92be89e-02de-4bb8-a697-9baaf95cd860 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:10.320604 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92be89e-02de-4bb8-a697-9baaf95cd860 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:10.321072 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 612/1219] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Nov 2 07:29:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:29:10.330179 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25494c58-ebcc-4cdb-8005-08768a480847 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:10.342419 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25494c58-ebcc-4cdb-8005-08768a480847 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:10.353977 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-c92be89e-02de-4bb8-a697-9baaf95cd860 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:29:10.355800 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-c92be89e-02de-4bb8-a697-9baaf95cd860 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:10.357429 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b9f7fc-5023-43a4-89b6-e09edb9b2b5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:10.358514 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b9f7fc-5023-43a4-89b6-e09edb9b2b5f admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:10.359164 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b9f7fc-5023-43a4-89b6-e09edb9b2b5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:10.359683 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b9f7fc-5023-43a4-89b6-e09edb9b2b5f admin admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:10.360558 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b9f7fc-5023-43a4-89b6-e09edb9b2b5f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:10.362774 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b9f7fc-5023-43a4-89b6-e09edb9b2b5f admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:10.371019 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 608/1220] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:29:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:29:10.381702 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b604a3-1733-4b45-bc7d-71d07a5e829e None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:10.382074 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b604a3-1733-4b45-bc7d-71d07a5e829e None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:10.382286 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b604a3-1733-4b45-bc7d-71d07a5e829e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:10.419576 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-f4b604a3-1733-4b45-bc7d-71d07a5e829e None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:29:10.420815 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-f4b604a3-1733-4b45-bc7d-71d07a5e829e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:10.436136 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 613/1221] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Nov 2 07:29:10 2022] POST /identity/v3/projects => generated 391 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 07:29:10.441251 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 609/1222] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:29:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:29:10.441774 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b9f7fc-5023-43a4-89b6-e09edb9b2b5f admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:10.445910 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b604a3-1733-4b45-bc7d-71d07a5e829e None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:10.466404 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b604a3-1733-4b45-bc7d-71d07a5e829e None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:10.551574 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b9f7fc-5023-43a4-89b6-e09edb9b2b5f admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:10.564185 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1540f703-615a-4d86-b6e2-dab5278d2ec2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:10.565258 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1540f703-615a-4d86-b6e2-dab5278d2ec2 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:10.565772 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1540f703-615a-4d86-b6e2-dab5278d2ec2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:10.566303 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1540f703-615a-4d86-b6e2-dab5278d2ec2 admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:10.569317 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1540f703-615a-4d86-b6e2-dab5278d2ec2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:10.571865 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: WARNING py.warnings [None req-1540f703-615a-4d86-b6e2-dab5278d2ec2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Nov 02 07:29:10.571865 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: warnings.warn(msg) Nov 02 07:29:10.571865 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: Nov 02 07:29:10.573645 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1540f703-615a-4d86-b6e2-dab5278d2ec2 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:10.589134 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c23ad4e-e254-4ddb-8e42-937d8caad045 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:10.590883 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c23ad4e-e254-4ddb-8e42-937d8caad045 admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:10.591416 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c23ad4e-e254-4ddb-8e42-937d8caad045 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:10.591910 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c23ad4e-e254-4ddb-8e42-937d8caad045 admin admin] PATH_INFO: `/v3/domains` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:10.592819 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c23ad4e-e254-4ddb-8e42-937d8caad045 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:29:10.593362 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c23ad4e-e254-4ddb-8e42-937d8caad045 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:10.595271 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c23ad4e-e254-4ddb-8e42-937d8caad045 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:10.625082 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 610/1223] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Nov 2 07:29:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:29:10.626790 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0b14fe-3f2e-46b0-8bfe-bcc0ba807ba6 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:10.627303 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0b14fe-3f2e-46b0-8bfe-bcc0ba807ba6 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:10.627807 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0b14fe-3f2e-46b0-8bfe-bcc0ba807ba6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:10.638892 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 614/1224] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Nov 2 07:29:10 2022] POST /identity/v3/users => generated 296 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:29:10.642750 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa8a65c-3206-4470-8ceb-e926a0d9d75a None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:10.643278 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa8a65c-3206-4470-8ceb-e926a0d9d75a None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:10.643798 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa8a65c-3206-4470-8ceb-e926a0d9d75a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:10.700238 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-7c0b14fe-3f2e-46b0-8bfe-bcc0ba807ba6 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:29:10.701821 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-7c0b14fe-3f2e-46b0-8bfe-bcc0ba807ba6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:10.702884 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-6fa8a65c-3206-4470-8ceb-e926a0d9d75a None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:29:10.706384 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-6fa8a65c-3206-4470-8ceb-e926a0d9d75a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:10.724840 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 615/1225] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:29:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:29:10.725479 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 611/1226] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:29:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:29:10.725722 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa8a65c-3206-4470-8ceb-e926a0d9d75a None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:10.726846 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0b14fe-3f2e-46b0-8bfe-bcc0ba807ba6 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:10.739197 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa8a65c-3206-4470-8ceb-e926a0d9d75a None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:10.747068 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0b14fe-3f2e-46b0-8bfe-bcc0ba807ba6 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:10.753895 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49efa6f-ccde-4cd2-bb36-2c3ba1bc5df6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:10.754425 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49efa6f-ccde-4cd2-bb36-2c3ba1bc5df6 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:10.754706 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49efa6f-ccde-4cd2-bb36-2c3ba1bc5df6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:10.754976 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49efa6f-ccde-4cd2-bb36-2c3ba1bc5df6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:10.755610 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49efa6f-ccde-4cd2-bb36-2c3ba1bc5df6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:10.758302 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49efa6f-ccde-4cd2-bb36-2c3ba1bc5df6 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:10.762270 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f788c04f-c0ab-477a-85c5-812b13cc2196 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:10.762958 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f788c04f-c0ab-477a-85c5-812b13cc2196 admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:10.763236 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f788c04f-c0ab-477a-85c5-812b13cc2196 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:10.764152 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f788c04f-c0ab-477a-85c5-812b13cc2196 admin admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:10.764860 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f788c04f-c0ab-477a-85c5-812b13cc2196 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:10.766551 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f788c04f-c0ab-477a-85c5-812b13cc2196 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:10.784805 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 612/1227] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:29:10 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:29:10.786587 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56ebf68-60b1-4700-afc0-9a539c53058f None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:10.787105 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56ebf68-60b1-4700-afc0-9a539c53058f None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:10.787604 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56ebf68-60b1-4700-afc0-9a539c53058f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:10.813578 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 616/1228] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Nov 2 07:29:10 2022] POST /identity/v3/projects => generated 371 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:29:10.815186 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce60aa86-aebb-4b63-8617-8cf498220e3f None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:10.815524 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce60aa86-aebb-4b63-8617-8cf498220e3f None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:10.815801 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce60aa86-aebb-4b63-8617-8cf498220e3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:10.818062 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-e56ebf68-60b1-4700-afc0-9a539c53058f None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:29:10.819498 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-e56ebf68-60b1-4700-afc0-9a539c53058f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:10.837559 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-ce60aa86-aebb-4b63-8617-8cf498220e3f None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:29:10.838808 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-ce60aa86-aebb-4b63-8617-8cf498220e3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:10.883134 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 613/1229] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:29:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:29:10.884370 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eeaab36-cc3d-46dc-b62b-dd9d18f79212 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:10.884650 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eeaab36-cc3d-46dc-b62b-dd9d18f79212 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:10.884919 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eeaab36-cc3d-46dc-b62b-dd9d18f79212 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:10.908210 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 617/1230] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:29:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:29:10.909122 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce60aa86-aebb-4b63-8617-8cf498220e3f None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:10.915718 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-6eeaab36-cc3d-46dc-b62b-dd9d18f79212 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:29:10.916927 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-6eeaab36-cc3d-46dc-b62b-dd9d18f79212 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:10.918411 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce60aa86-aebb-4b63-8617-8cf498220e3f None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:10.922928 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6117cc-8db0-4bc5-8b74-3697ea361098 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:10.923515 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6117cc-8db0-4bc5-8b74-3697ea361098 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:10.923794 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6117cc-8db0-4bc5-8b74-3697ea361098 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:10.924067 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6117cc-8db0-4bc5-8b74-3697ea361098 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:10.924731 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6117cc-8db0-4bc5-8b74-3697ea361098 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:10.924830 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 614/1231] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:29:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:29:10.925972 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ad8b25-917b-4788-90bf-c04a50a98ddd None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:10.926254 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ad8b25-917b-4788-90bf-c04a50a98ddd None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:10.926352 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6117cc-8db0-4bc5-8b74-3697ea361098 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:10.926561 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ad8b25-917b-4788-90bf-c04a50a98ddd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:10.932823 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:29:10.939508 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 618/1232] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:29:10 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:29:10.941624 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5201356-aa11-407c-afde-5c4b7faa26d1 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:10.942274 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5201356-aa11-407c-afde-5c4b7faa26d1 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:10.942675 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5201356-aa11-407c-afde-5c4b7faa26d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:10.946823 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-b3ad8b25-917b-4788-90bf-c04a50a98ddd None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:29:10.948019 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-b3ad8b25-917b-4788-90bf-c04a50a98ddd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:10.956817 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 615/1233] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:29:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:29:10.957635 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ad8b25-917b-4788-90bf-c04a50a98ddd None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:10.963308 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-a5201356-aa11-407c-afde-5c4b7faa26d1 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:29:10.965126 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-a5201356-aa11-407c-afde-5c4b7faa26d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:10.969186 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ad8b25-917b-4788-90bf-c04a50a98ddd None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:10.973156 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 619/1234] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:29:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:29:10.973988 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760c7180-cb1c-4921-82d2-61a15c673781 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:10.974616 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760c7180-cb1c-4921-82d2-61a15c673781 admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:10.974893 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760c7180-cb1c-4921-82d2-61a15c673781 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:10.975161 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760c7180-cb1c-4921-82d2-61a15c673781 admin admin] PATH_INFO: `/v3/users` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:10.975805 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760c7180-cb1c-4921-82d2-61a15c673781 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:10.977692 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760c7180-cb1c-4921-82d2-61a15c673781 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:10.979704 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b1c326-b14d-4a71-b952-caac5b163f8c None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:10.980139 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b1c326-b14d-4a71-b952-caac5b163f8c None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:10.980510 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b1c326-b14d-4a71-b952-caac5b163f8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:11.001020 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 616/1235] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Nov 2 07:29:10 2022] POST /identity/v3/users => generated 286 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:29:11.001835 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760c7180-cb1c-4921-82d2-61a15c673781 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:11.003988 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-18b1c326-b14d-4a71-b952-caac5b163f8c None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:29:11.005096 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-18b1c326-b14d-4a71-b952-caac5b163f8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:11.011812 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760c7180-cb1c-4921-82d2-61a15c673781 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:11.013850 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 620/1236] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:29:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:29:11.015056 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82516f5f-86de-4ad4-b18c-fd61b6452bcd None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:11.015341 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82516f5f-86de-4ad4-b18c-fd61b6452bcd None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:11.015607 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82516f5f-86de-4ad4-b18c-fd61b6452bcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:11.016626 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dd4176-99a1-4004-8c43-899f204838da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:11.017213 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dd4176-99a1-4004-8c43-899f204838da admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:11.017491 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dd4176-99a1-4004-8c43-899f204838da admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:11.017759 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dd4176-99a1-4004-8c43-899f204838da admin admin] PATH_INFO: `/v3/domains` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:11.018481 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7dd4176-99a1-4004-8c43-899f204838da admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:29:11.018720 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7dd4176-99a1-4004-8c43-899f204838da admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:11.020293 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7dd4176-99a1-4004-8c43-899f204838da admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:11.033717 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 617/1237] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Nov 2 07:29:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:29:11.034637 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dd4176-99a1-4004-8c43-899f204838da admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:11.035287 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-82516f5f-86de-4ad4-b18c-fd61b6452bcd None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:29:11.036473 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-82516f5f-86de-4ad4-b18c-fd61b6452bcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:11.044897 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 621/1238] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:29:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:29:11.045778 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82516f5f-86de-4ad4-b18c-fd61b6452bcd None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:11.046195 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dd4176-99a1-4004-8c43-899f204838da admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:11.051697 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481e4107-e738-43c0-90e3-1f8998d606b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:11.052354 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481e4107-e738-43c0-90e3-1f8998d606b7 admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:11.052637 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481e4107-e738-43c0-90e3-1f8998d606b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:11.052908 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481e4107-e738-43c0-90e3-1f8998d606b7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:11.053542 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481e4107-e738-43c0-90e3-1f8998d606b7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:29:11.053843 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481e4107-e738-43c0-90e3-1f8998d606b7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:11.055541 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481e4107-e738-43c0-90e3-1f8998d606b7 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:11.059723 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82516f5f-86de-4ad4-b18c-fd61b6452bcd None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:11.065000 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:29:11.065342 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc72ffc-a697-4e58-8a60-27ffe9505cfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:11.066383 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc72ffc-a697-4e58-8a60-27ffe9505cfe admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:11.066733 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc72ffc-a697-4e58-8a60-27ffe9505cfe admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:11.067037 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc72ffc-a697-4e58-8a60-27ffe9505cfe admin admin] PATH_INFO: `/v3/projects/c60cb95cbf734d9b852af2bdcad4e1c0/users/0e571b0f4d6f4fafb71939f501b8deec/roles/b494fedd00f24f0b84a529e8672a156a` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:11.070636 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 618/1239] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Nov 2 07:29:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:29:11.072199 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1214af3f-079e-4e40-aa2a-150d9967ce8a None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:11.072695 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1214af3f-079e-4e40-aa2a-150d9967ce8a None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:11.073139 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1214af3f-079e-4e40-aa2a-150d9967ce8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:11.089102 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc72ffc-a697-4e58-8a60-27ffe9505cfe admin admin] RBAC: Authorizing `identity:create_grant(project_id=c60cb95cbf734d9b852af2bdcad4e1c0, user_id=0e571b0f4d6f4fafb71939f501b8deec, role_id=b494fedd00f24f0b84a529e8672a156a)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:11.092114 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc72ffc-a697-4e58-8a60-27ffe9505cfe admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:11.092636 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-1214af3f-079e-4e40-aa2a-150d9967ce8a None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:29:11.093843 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-1214af3f-079e-4e40-aa2a-150d9967ce8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:11.101990 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 619/1240] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:29:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:29:11.103232 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2c282e-f425-4dfe-9f4d-d7f5e76eae65 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:11.103483 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2c282e-f425-4dfe-9f4d-d7f5e76eae65 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:11.103746 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2c282e-f425-4dfe-9f4d-d7f5e76eae65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:11.109682 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 622/1241] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Nov 2 07:29:11 2022] PUT /identity/v3/projects/c60cb95cbf734d9b852af2bdcad4e1c0/users/0e571b0f4d6f4fafb71939f501b8deec/roles/b494fedd00f24f0b84a529e8672a156a => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:29:11.110622 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc72ffc-a697-4e58-8a60-27ffe9505cfe admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:11.125937 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-1b2c282e-f425-4dfe-9f4d-d7f5e76eae65 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:29:11.127289 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-1b2c282e-f425-4dfe-9f4d-d7f5e76eae65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:11.172523 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc72ffc-a697-4e58-8a60-27ffe9505cfe admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:11.177227 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f4bf87-56a8-4009-9ba4-ae32b422218a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:11.177833 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f4bf87-56a8-4009-9ba4-ae32b422218a admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:11.178100 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f4bf87-56a8-4009-9ba4-ae32b422218a admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:11.178381 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f4bf87-56a8-4009-9ba4-ae32b422218a admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:11.179007 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f4bf87-56a8-4009-9ba4-ae32b422218a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:11.180571 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f4bf87-56a8-4009-9ba4-ae32b422218a admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:11.188308 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 620/1242] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:29:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:29:11.194943 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea4df89-20d0-4f8a-a028-aaa3c9930e91 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:11.194943 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea4df89-20d0-4f8a-a028-aaa3c9930e91 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:11.194943 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea4df89-20d0-4f8a-a028-aaa3c9930e91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:11.195241 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 623/1243] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:29:11 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:29:11.195241 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f4bf87-56a8-4009-9ba4-ae32b422218a admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:11.204000 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f4bf87-56a8-4009-9ba4-ae32b422218a admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:11.208829 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07684169-d0ba-4afb-9ead-72fc4204b245 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:11.209356 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07684169-d0ba-4afb-9ead-72fc4204b245 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:11.209759 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07684169-d0ba-4afb-9ead-72fc4204b245 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:11.210035 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07684169-d0ba-4afb-9ead-72fc4204b245 admin admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:11.210696 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07684169-d0ba-4afb-9ead-72fc4204b245 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:11.211710 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-dea4df89-20d0-4f8a-a028-aaa3c9930e91 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:29:11.212624 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07684169-d0ba-4afb-9ead-72fc4204b245 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:11.212880 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-dea4df89-20d0-4f8a-a028-aaa3c9930e91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:11.220645 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 621/1244] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:29:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:29:11.224442 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea4df89-20d0-4f8a-a028-aaa3c9930e91 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:11.234941 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea4df89-20d0-4f8a-a028-aaa3c9930e91 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:11.239669 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b903bb31-2072-4403-b3d3-88020cbe8795 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:11.240261 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b903bb31-2072-4403-b3d3-88020cbe8795 admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:11.240540 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b903bb31-2072-4403-b3d3-88020cbe8795 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:11.240808 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b903bb31-2072-4403-b3d3-88020cbe8795 admin admin] PATH_INFO: `/v3/domains` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:11.241691 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b903bb31-2072-4403-b3d3-88020cbe8795 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:29:11.241847 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 624/1245] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Nov 2 07:29:11 2022] POST /identity/v3/projects => generated 379 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:29:11.242342 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b903bb31-2072-4403-b3d3-88020cbe8795 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:11.242735 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07684169-d0ba-4afb-9ead-72fc4204b245 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:11.244093 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b903bb31-2072-4403-b3d3-88020cbe8795 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:11.256872 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 622/1246] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Nov 2 07:29:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:29:11.264517 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b903bb31-2072-4403-b3d3-88020cbe8795 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:11.268429 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b903bb31-2072-4403-b3d3-88020cbe8795 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:11.305094 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07684169-d0ba-4afb-9ead-72fc4204b245 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:11.310021 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3413edd1-00d4-4724-a29c-13747061912b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:11.310658 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3413edd1-00d4-4724-a29c-13747061912b admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:11.310941 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3413edd1-00d4-4724-a29c-13747061912b admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:11.311220 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3413edd1-00d4-4724-a29c-13747061912b admin admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:11.311839 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3413edd1-00d4-4724-a29c-13747061912b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:11.313769 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3413edd1-00d4-4724-a29c-13747061912b admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:11.323181 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f458372f-f747-4855-a66a-9012ba0a98f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:11.323893 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f458372f-f747-4855-a66a-9012ba0a98f1 admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:11.324243 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f458372f-f747-4855-a66a-9012ba0a98f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:11.324591 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f458372f-f747-4855-a66a-9012ba0a98f1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:11.325222 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f458372f-f747-4855-a66a-9012ba0a98f1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:29:11.325519 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f458372f-f747-4855-a66a-9012ba0a98f1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:11.327140 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f458372f-f747-4855-a66a-9012ba0a98f1 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:11.341117 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 623/1247] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Nov 2 07:29:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:29:11.341117 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f445aa2-66de-4966-84ca-acac2eafdba6 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:11.341117 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f445aa2-66de-4966-84ca-acac2eafdba6 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:11.341117 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f445aa2-66de-4966-84ca-acac2eafdba6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:11.349701 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 625/1248] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Nov 2 07:29:11 2022] POST /identity/v3/projects => generated 365 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:29:11.350874 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3413edd1-00d4-4724-a29c-13747061912b admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:11.364795 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-7f445aa2-66de-4966-84ca-acac2eafdba6 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:29:11.366302 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-7f445aa2-66de-4966-84ca-acac2eafdba6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:11.370269 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:29:11.416430 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3413edd1-00d4-4724-a29c-13747061912b admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:11.421693 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84450b9-8318-4c43-80dc-3dda47277fc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:11.422496 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84450b9-8318-4c43-80dc-3dda47277fc5 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:11.422934 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84450b9-8318-4c43-80dc-3dda47277fc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:11.423216 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84450b9-8318-4c43-80dc-3dda47277fc5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:11.423844 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84450b9-8318-4c43-80dc-3dda47277fc5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:11.425410 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84450b9-8318-4c43-80dc-3dda47277fc5 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:11.426883 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 624/1249] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:29:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:29:11.427723 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f445aa2-66de-4966-84ca-acac2eafdba6 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:11.435895 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 626/1250] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:29:11 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:29:11.436794 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84450b9-8318-4c43-80dc-3dda47277fc5 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:11.437391 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f445aa2-66de-4966-84ca-acac2eafdba6 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:11.442709 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51dc39e1-fcb9-4adf-8023-9d28e7c27ef6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:11.443339 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dc39e1-fcb9-4adf-8023-9d28e7c27ef6 admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:11.443611 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dc39e1-fcb9-4adf-8023-9d28e7c27ef6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:11.443898 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dc39e1-fcb9-4adf-8023-9d28e7c27ef6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:11.444528 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51dc39e1-fcb9-4adf-8023-9d28e7c27ef6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:29:11.444823 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51dc39e1-fcb9-4adf-8023-9d28e7c27ef6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:11.446460 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51dc39e1-fcb9-4adf-8023-9d28e7c27ef6 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:11.448030 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84450b9-8318-4c43-80dc-3dda47277fc5 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:11.448909 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:29:11.453042 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1625f5ea-58c0-42a3-a689-51df8cfd82e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:11.453632 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1625f5ea-58c0-42a3-a689-51df8cfd82e5 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:11.453916 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1625f5ea-58c0-42a3-a689-51df8cfd82e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:11.454186 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1625f5ea-58c0-42a3-a689-51df8cfd82e5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:11.454899 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1625f5ea-58c0-42a3-a689-51df8cfd82e5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:29:11.455196 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1625f5ea-58c0-42a3-a689-51df8cfd82e5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:11.456759 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1625f5ea-58c0-42a3-a689-51df8cfd82e5 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:11.459290 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 625/1251] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Nov 2 07:29:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:29:11.460088 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51dc39e1-fcb9-4adf-8023-9d28e7c27ef6 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:11.469810 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 627/1252] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Nov 2 07:29:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:29:11.470632 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1625f5ea-58c0-42a3-a689-51df8cfd82e5 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:11.477811 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51dc39e1-fcb9-4adf-8023-9d28e7c27ef6 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:11.482055 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1625f5ea-58c0-42a3-a689-51df8cfd82e5 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:11.483254 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bab3542-9056-4b4b-87fc-1dd31cf61639 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:11.483978 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bab3542-9056-4b4b-87fc-1dd31cf61639 admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:11.484231 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bab3542-9056-4b4b-87fc-1dd31cf61639 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:11.484533 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bab3542-9056-4b4b-87fc-1dd31cf61639 admin admin] PATH_INFO: `/v3/users` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:11.485150 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bab3542-9056-4b4b-87fc-1dd31cf61639 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:11.486835 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26307e4c-3017-4b75-bca7-ee96c6884848 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:11.487210 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bab3542-9056-4b4b-87fc-1dd31cf61639 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:11.487408 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26307e4c-3017-4b75-bca7-ee96c6884848 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:11.487685 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26307e4c-3017-4b75-bca7-ee96c6884848 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:11.487953 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26307e4c-3017-4b75-bca7-ee96c6884848 admin admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:11.488560 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26307e4c-3017-4b75-bca7-ee96c6884848 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:11.490459 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26307e4c-3017-4b75-bca7-ee96c6884848 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:11.509991 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 626/1253] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Nov 2 07:29:11 2022] POST /identity/v3/users => generated 290 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:29:11.510914 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bab3542-9056-4b4b-87fc-1dd31cf61639 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:11.517490 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 628/1254] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Nov 2 07:29:11 2022] POST /identity/v3/projects => generated 367 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:29:11.518276 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26307e4c-3017-4b75-bca7-ee96c6884848 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:11.522968 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bab3542-9056-4b4b-87fc-1dd31cf61639 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:11.580457 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea05d1bb-8fcd-44b2-8b4f-bdc3f974efc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:11.581061 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea05d1bb-8fcd-44b2-8b4f-bdc3f974efc1 admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:11.581342 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea05d1bb-8fcd-44b2-8b4f-bdc3f974efc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:11.581609 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea05d1bb-8fcd-44b2-8b4f-bdc3f974efc1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:11.582221 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea05d1bb-8fcd-44b2-8b4f-bdc3f974efc1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:11.582727 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26307e4c-3017-4b75-bca7-ee96c6884848 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:11.583310 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: WARNING py.warnings [None req-ea05d1bb-8fcd-44b2-8b4f-bdc3f974efc1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Nov 02 07:29:11.583310 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: warnings.warn(msg) Nov 02 07:29:11.583310 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: Nov 02 07:29:11.584487 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea05d1bb-8fcd-44b2-8b4f-bdc3f974efc1 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:11.587332 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82982954-9b00-4303-9d20-0d070f708756 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:11.587930 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82982954-9b00-4303-9d20-0d070f708756 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:11.588210 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82982954-9b00-4303-9d20-0d070f708756 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:11.588477 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82982954-9b00-4303-9d20-0d070f708756 admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:11.589095 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82982954-9b00-4303-9d20-0d070f708756 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:11.593058 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82982954-9b00-4303-9d20-0d070f708756 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:11.612406 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 627/1255] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Nov 2 07:29:11 2022] POST /identity/v3/projects => generated 383 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:29:11.613283 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea05d1bb-8fcd-44b2-8b4f-bdc3f974efc1 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:11.615475 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 629/1256] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Nov 2 07:29:11 2022] POST /identity/v3/users => generated 283 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:29:11.616710 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82982954-9b00-4303-9d20-0d070f708756 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:11.629181 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82982954-9b00-4303-9d20-0d070f708756 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:11.680485 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea05d1bb-8fcd-44b2-8b4f-bdc3f974efc1 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:11.684862 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f28c541-234c-4d8b-8586-bfcd861a7dfb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3ec6ecb85f94ef59762877d9ea33ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:11.685474 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f28c541-234c-4d8b-8586-bfcd861a7dfb service neutron] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:11.685752 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f28c541-234c-4d8b-8586-bfcd861a7dfb service neutron] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:11.686088 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f28c541-234c-4d8b-8586-bfcd861a7dfb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:11.690876 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cf83f1-6978-4382-999f-d8874f169616 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:11.691568 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cf83f1-6978-4382-999f-d8874f169616 admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:11.691845 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cf83f1-6978-4382-999f-d8874f169616 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:11.692143 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cf83f1-6978-4382-999f-d8874f169616 admin admin] PATH_INFO: `/v3/projects/948d495ca7dc4cc4a2b8cb2e2d621324/users/72e9291d458043139a23efef6ff415ea/roles/b494fedd00f24f0b84a529e8672a156a` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:11.703173 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:29:11.707134 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cf83f1-6978-4382-999f-d8874f169616 admin admin] RBAC: Authorizing `identity:create_grant(project_id=948d495ca7dc4cc4a2b8cb2e2d621324, user_id=72e9291d458043139a23efef6ff415ea, role_id=b494fedd00f24f0b84a529e8672a156a)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:11.710107 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cf83f1-6978-4382-999f-d8874f169616 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:11.724966 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 630/1257] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Nov 2 07:29:11 2022] PUT /identity/v3/projects/948d495ca7dc4cc4a2b8cb2e2d621324/users/72e9291d458043139a23efef6ff415ea/roles/b494fedd00f24f0b84a529e8672a156a => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:29:11.725820 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cf83f1-6978-4382-999f-d8874f169616 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:11.751945 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f28c541-234c-4d8b-8586-bfcd861a7dfb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:11.753392 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f28c541-234c-4d8b-8586-bfcd861a7dfb service neutron] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:11.763234 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 628/1258] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:29:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:29:11.764005 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f28c541-234c-4d8b-8586-bfcd861a7dfb service neutron] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:11.774152 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f28c541-234c-4d8b-8586-bfcd861a7dfb service neutron] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:11.783894 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:29:11.795867 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cf83f1-6978-4382-999f-d8874f169616 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:11.801183 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92142685-d31e-4210-8196-5e276e8a21f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:11.801900 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92142685-d31e-4210-8196-5e276e8a21f0 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:11.802273 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92142685-d31e-4210-8196-5e276e8a21f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:11.802689 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92142685-d31e-4210-8196-5e276e8a21f0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:11.803389 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92142685-d31e-4210-8196-5e276e8a21f0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:11.805350 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92142685-d31e-4210-8196-5e276e8a21f0 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:11.839213 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 631/1259] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Nov 2 07:29:11 2022] POST /identity/v3/projects => generated 405 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:29:11.839213 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92142685-d31e-4210-8196-5e276e8a21f0 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:11.842018 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43e9f95-5474-4edd-b65a-d435760d3a07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:11.842804 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43e9f95-5474-4edd-b65a-d435760d3a07 admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:11.843249 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43e9f95-5474-4edd-b65a-d435760d3a07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:11.844172 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43e9f95-5474-4edd-b65a-d435760d3a07 admin admin] PATH_INFO: `/v3/projects` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:11.844826 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43e9f95-5474-4edd-b65a-d435760d3a07 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:11.846877 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43e9f95-5474-4edd-b65a-d435760d3a07 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:11.879446 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 629/1260] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Nov 2 07:29:11 2022] POST /identity/v3/projects => generated 387 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:29:11.880538 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43e9f95-5474-4edd-b65a-d435760d3a07 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:11.912193 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92142685-d31e-4210-8196-5e276e8a21f0 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:11.952169 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43e9f95-5474-4edd-b65a-d435760d3a07 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:11.957407 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbcacbb-2a19-4a07-88b1-bd0e0c032a7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:11.958181 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbcacbb-2a19-4a07-88b1-bd0e0c032a7d admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:11.958580 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbcacbb-2a19-4a07-88b1-bd0e0c032a7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:11.959060 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbcacbb-2a19-4a07-88b1-bd0e0c032a7d admin admin] PATH_INFO: `/v3/users` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:11.959787 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbcacbb-2a19-4a07-88b1-bd0e0c032a7d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:11.962462 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbcacbb-2a19-4a07-88b1-bd0e0c032a7d admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:11.993572 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 630/1261] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Nov 2 07:29:11 2022] POST /identity/v3/users => generated 284 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:29:11.993572 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbcacbb-2a19-4a07-88b1-bd0e0c032a7d admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:11.999056 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e02b56-8483-4b92-abab-e96d4925f836 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:11.999835 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e02b56-8483-4b92-abab-e96d4925f836 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:12.000211 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e02b56-8483-4b92-abab-e96d4925f836 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:12.000585 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e02b56-8483-4b92-abab-e96d4925f836 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:12.001876 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e02b56-8483-4b92-abab-e96d4925f836 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:12.003638 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e02b56-8483-4b92-abab-e96d4925f836 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:12.004942 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbcacbb-2a19-4a07-88b1-bd0e0c032a7d admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:12.010011 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0508fee-c86e-40b9-a790-bcbcc5ecf4f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:12.010637 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0508fee-c86e-40b9-a790-bcbcc5ecf4f4 admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:12.010917 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0508fee-c86e-40b9-a790-bcbcc5ecf4f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:12.011225 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0508fee-c86e-40b9-a790-bcbcc5ecf4f4 admin admin] PATH_INFO: `/v3/users` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:12.012115 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0508fee-c86e-40b9-a790-bcbcc5ecf4f4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:12.014013 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0508fee-c86e-40b9-a790-bcbcc5ecf4f4 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:12.018503 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 632/1262] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:29:11 2022] GET /identity/v3/roles => generated 1420 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:29:12.018503 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e02b56-8483-4b92-abab-e96d4925f836 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:12.029502 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e02b56-8483-4b92-abab-e96d4925f836 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:12.036579 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c606d6aa-6f5a-4cd7-a5ae-0fc292f25865 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:12.037387 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c606d6aa-6f5a-4cd7-a5ae-0fc292f25865 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:12.038026 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c606d6aa-6f5a-4cd7-a5ae-0fc292f25865 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:12.038469 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c606d6aa-6f5a-4cd7-a5ae-0fc292f25865 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:12.039176 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c606d6aa-6f5a-4cd7-a5ae-0fc292f25865 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:12.040945 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 631/1263] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Nov 2 07:29:11 2022] POST /identity/v3/users => generated 292 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:29:12.041104 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c606d6aa-6f5a-4cd7-a5ae-0fc292f25865 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:12.045253 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1feab21b-cddc-4d35-b005-b0b0b60b7782 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:12.045253 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1feab21b-cddc-4d35-b005-b0b0b60b7782 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:12.045253 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1feab21b-cddc-4d35-b005-b0b0b60b7782 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:12.054407 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 633/1264] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:29:12 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:29:12.054407 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c606d6aa-6f5a-4cd7-a5ae-0fc292f25865 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:12.068801 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c606d6aa-6f5a-4cd7-a5ae-0fc292f25865 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:12.075328 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735acc48-8d7a-43f7-9893-b8287013dcdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:12.075554 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-1feab21b-cddc-4d35-b005-b0b0b60b7782 None None] MFA Rules not processed for user `72e9291d458043139a23efef6ff415ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:29:12.076150 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735acc48-8d7a-43f7-9893-b8287013dcdd admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:12.076536 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735acc48-8d7a-43f7-9893-b8287013dcdd admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:12.076762 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-1feab21b-cddc-4d35-b005-b0b0b60b7782 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:12.077012 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735acc48-8d7a-43f7-9893-b8287013dcdd admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:12.077738 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735acc48-8d7a-43f7-9893-b8287013dcdd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:12.080203 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735acc48-8d7a-43f7-9893-b8287013dcdd admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:12.091692 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:29:12.112863 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 634/1265] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Nov 2 07:29:12 2022] POST /identity/v3/users => generated 303 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:29:12.114610 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735acc48-8d7a-43f7-9893-b8287013dcdd admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:12.126171 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735acc48-8d7a-43f7-9893-b8287013dcdd admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:12.131900 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc97b3be-75b1-461a-8328-a28ef532cc9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:12.132511 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc97b3be-75b1-461a-8328-a28ef532cc9e admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:12.132787 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc97b3be-75b1-461a-8328-a28ef532cc9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:12.133081 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc97b3be-75b1-461a-8328-a28ef532cc9e admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:12.133707 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc97b3be-75b1-461a-8328-a28ef532cc9e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:12.135926 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc97b3be-75b1-461a-8328-a28ef532cc9e admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:12.152521 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:29:12.157633 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 632/1266] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:29:12 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:29:12.158770 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1feab21b-cddc-4d35-b005-b0b0b60b7782 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:12.165788 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 635/1267] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Nov 2 07:29:12 2022] POST /identity/v3/users => generated 294 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:29:12.166643 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc97b3be-75b1-461a-8328-a28ef532cc9e admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:12.178110 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc97b3be-75b1-461a-8328-a28ef532cc9e admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:12.180445 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1feab21b-cddc-4d35-b005-b0b0b60b7782 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:12.183895 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faea4be0-3a56-4f63-94e1-efbb9e777bc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:12.184515 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faea4be0-3a56-4f63-94e1-efbb9e777bc1 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:12.184795 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faea4be0-3a56-4f63-94e1-efbb9e777bc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:12.185066 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faea4be0-3a56-4f63-94e1-efbb9e777bc1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:12.185705 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faea4be0-3a56-4f63-94e1-efbb9e777bc1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:12.187655 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faea4be0-3a56-4f63-94e1-efbb9e777bc1 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:12.188184 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23397203-2d84-414b-91dd-9a707c185692 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:12.189914 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23397203-2d84-414b-91dd-9a707c185692 admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:12.191832 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23397203-2d84-414b-91dd-9a707c185692 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:12.192370 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23397203-2d84-414b-91dd-9a707c185692 admin admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:12.193102 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23397203-2d84-414b-91dd-9a707c185692 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:12.195156 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23397203-2d84-414b-91dd-9a707c185692 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:12.214278 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 633/1268] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:29:12 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:29:12.214278 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23397203-2d84-414b-91dd-9a707c185692 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:12.219077 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 636/1269] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:29:12 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:29:12.220117 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faea4be0-3a56-4f63-94e1-efbb9e777bc1 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:12.233066 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faea4be0-3a56-4f63-94e1-efbb9e777bc1 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:12.238194 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69e85dc-2dc7-4819-8acc-79b8bd6d0502 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:12.239169 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69e85dc-2dc7-4819-8acc-79b8bd6d0502 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:12.239390 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23397203-2d84-414b-91dd-9a707c185692 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:12.240202 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69e85dc-2dc7-4819-8acc-79b8bd6d0502 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:12.240693 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69e85dc-2dc7-4819-8acc-79b8bd6d0502 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:12.241928 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69e85dc-2dc7-4819-8acc-79b8bd6d0502 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:12.244050 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69e85dc-2dc7-4819-8acc-79b8bd6d0502 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:12.247140 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e838a2-f6b3-4a65-bbe1-cc43dd7b2d17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:12.248016 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e838a2-f6b3-4a65-bbe1-cc43dd7b2d17 admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:12.248555 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e838a2-f6b3-4a65-bbe1-cc43dd7b2d17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:12.248918 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e838a2-f6b3-4a65-bbe1-cc43dd7b2d17 admin admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:12.249845 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e838a2-f6b3-4a65-bbe1-cc43dd7b2d17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:12.251666 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e838a2-f6b3-4a65-bbe1-cc43dd7b2d17 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:12.259848 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 637/1270] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:29:12 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:29:12.263740 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69e85dc-2dc7-4819-8acc-79b8bd6d0502 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:12.271383 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 634/1271] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:29:12 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:29:12.272321 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e838a2-f6b3-4a65-bbe1-cc43dd7b2d17 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:12.281044 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69e85dc-2dc7-4819-8acc-79b8bd6d0502 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:12.285176 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e838a2-f6b3-4a65-bbe1-cc43dd7b2d17 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:12.286264 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e23a739-aa47-468d-8d52-e1a8c62eefd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:12.286956 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e23a739-aa47-468d-8d52-e1a8c62eefd7 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:12.287231 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e23a739-aa47-468d-8d52-e1a8c62eefd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:12.287509 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e23a739-aa47-468d-8d52-e1a8c62eefd7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:12.288154 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e23a739-aa47-468d-8d52-e1a8c62eefd7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:12.289717 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e23a739-aa47-468d-8d52-e1a8c62eefd7 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:12.291069 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bbcda0-a56f-498e-9db4-20c4af0be86d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:12.291744 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bbcda0-a56f-498e-9db4-20c4af0be86d admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:12.292462 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bbcda0-a56f-498e-9db4-20c4af0be86d admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:12.293704 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bbcda0-a56f-498e-9db4-20c4af0be86d admin admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:12.294183 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bbcda0-a56f-498e-9db4-20c4af0be86d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:12.295830 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bbcda0-a56f-498e-9db4-20c4af0be86d admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:12.300970 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 638/1272] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:29:12 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:29:12.303997 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e9d7ba-5ea8-48f6-8cab-92c608cfa9fa None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:12.303997 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e9d7ba-5ea8-48f6-8cab-92c608cfa9fa None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:12.303997 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e9d7ba-5ea8-48f6-8cab-92c608cfa9fa None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:12.304720 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 639/1273] 10.222.0.18 () {54 vars in 961 bytes} [Wed Nov 2 07:29:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:29:12.307636 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e9d7ba-5ea8-48f6-8cab-92c608cfa9fa None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:12.310911 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 635/1274] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:29:12 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:29:12.311896 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bbcda0-a56f-498e-9db4-20c4af0be86d admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:12.322388 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bbcda0-a56f-498e-9db4-20c4af0be86d admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:12.327180 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e9d7ba-5ea8-48f6-8cab-92c608cfa9fa None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:12.328551 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558c74b1-dc9c-4fc6-8cf8-9fba988c9813 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:12.329221 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558c74b1-dc9c-4fc6-8cf8-9fba988c9813 admin admin] REQUEST_METHOD: `PUT` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:12.329506 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558c74b1-dc9c-4fc6-8cf8-9fba988c9813 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:12.329836 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558c74b1-dc9c-4fc6-8cf8-9fba988c9813 admin admin] PATH_INFO: `/v3/projects/f676ec82a80f4425a6fa7562ee47a4cf/users/17a6f8d01e6d433ca467238f07ec15ba/roles/b494fedd00f24f0b84a529e8672a156a` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:12.333606 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c209086-b97d-4322-887e-896a034c04c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:12.334378 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c209086-b97d-4322-887e-896a034c04c7 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:12.334958 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c209086-b97d-4322-887e-896a034c04c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:12.335111 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c209086-b97d-4322-887e-896a034c04c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:12.335738 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c209086-b97d-4322-887e-896a034c04c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:12.337384 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c209086-b97d-4322-887e-896a034c04c7 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:12.349081 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 640/1275] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:29:12 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:29:12.349851 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c209086-b97d-4322-887e-896a034c04c7 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:12.350068 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558c74b1-dc9c-4fc6-8cf8-9fba988c9813 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f676ec82a80f4425a6fa7562ee47a4cf, user_id=17a6f8d01e6d433ca467238f07ec15ba, role_id=b494fedd00f24f0b84a529e8672a156a)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:12.352974 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558c74b1-dc9c-4fc6-8cf8-9fba988c9813 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:12.360344 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c209086-b97d-4322-887e-896a034c04c7 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:12.365759 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681266ea-a031-4e99-b984-1c8dfd5d9454 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:12.366728 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681266ea-a031-4e99-b984-1c8dfd5d9454 admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:12.367008 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681266ea-a031-4e99-b984-1c8dfd5d9454 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:12.367308 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681266ea-a031-4e99-b984-1c8dfd5d9454 admin admin] PATH_INFO: `/v3/projects/818e1198e7c64603bb957c4ed7198b45/users/1bdbe843b0bd4718955df87196c1ec66/roles/b494fedd00f24f0b84a529e8672a156a` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:12.368771 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 636/1276] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Nov 2 07:29:12 2022] PUT /identity/v3/projects/f676ec82a80f4425a6fa7562ee47a4cf/users/17a6f8d01e6d433ca467238f07ec15ba/roles/b494fedd00f24f0b84a529e8672a156a => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:29:12.372029 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558c74b1-dc9c-4fc6-8cf8-9fba988c9813 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:12.385525 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681266ea-a031-4e99-b984-1c8dfd5d9454 admin admin] RBAC: Authorizing `identity:create_grant(project_id=818e1198e7c64603bb957c4ed7198b45, user_id=1bdbe843b0bd4718955df87196c1ec66, role_id=b494fedd00f24f0b84a529e8672a156a)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:12.388578 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681266ea-a031-4e99-b984-1c8dfd5d9454 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:12.402933 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 641/1277] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Nov 2 07:29:12 2022] PUT /identity/v3/projects/818e1198e7c64603bb957c4ed7198b45/users/1bdbe843b0bd4718955df87196c1ec66/roles/b494fedd00f24f0b84a529e8672a156a => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:29:12.403829 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681266ea-a031-4e99-b984-1c8dfd5d9454 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:12.451461 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558c74b1-dc9c-4fc6-8cf8-9fba988c9813 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:12.476754 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681266ea-a031-4e99-b984-1c8dfd5d9454 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:12.482312 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba57211-b6f5-4df9-85e9-4d5f36e3f673 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:12.483052 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba57211-b6f5-4df9-85e9-4d5f36e3f673 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:12.483405 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba57211-b6f5-4df9-85e9-4d5f36e3f673 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:12.483746 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba57211-b6f5-4df9-85e9-4d5f36e3f673 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:12.484471 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba57211-b6f5-4df9-85e9-4d5f36e3f673 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:12.486168 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba57211-b6f5-4df9-85e9-4d5f36e3f673 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:12.499165 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 642/1278] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:29:12 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:29:12.502534 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e275268-6458-42a7-a8fd-d24aca1a73c0 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:12.502534 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e275268-6458-42a7-a8fd-d24aca1a73c0 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:12.502534 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e275268-6458-42a7-a8fd-d24aca1a73c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:12.526289 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f016181-9161-401d-92b0-8e975d3378d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:12.527112 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f016181-9161-401d-92b0-8e975d3378d8 admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:12.527458 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f016181-9161-401d-92b0-8e975d3378d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:12.527803 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f016181-9161-401d-92b0-8e975d3378d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:12.528561 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f016181-9161-401d-92b0-8e975d3378d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:12.531206 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f016181-9161-401d-92b0-8e975d3378d8 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:12.543116 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 637/1279] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:29:12 2022] GET /identity/v3/roles => generated 1420 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:29:12.544607 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f016181-9161-401d-92b0-8e975d3378d8 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:12.547654 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-2e275268-6458-42a7-a8fd-d24aca1a73c0 None None] MFA Rules not processed for user `69e48cba3f804cf08b2477f01ae95179`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:29:12.549023 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-2e275268-6458-42a7-a8fd-d24aca1a73c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:12.555166 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f016181-9161-401d-92b0-8e975d3378d8 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:12.560896 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a818f23-d88c-4544-9f34-a11d0149a45c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:12.561574 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a818f23-d88c-4544-9f34-a11d0149a45c admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:12.561923 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a818f23-d88c-4544-9f34-a11d0149a45c admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:12.562287 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a818f23-d88c-4544-9f34-a11d0149a45c admin admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:12.563006 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a818f23-d88c-4544-9f34-a11d0149a45c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:12.564649 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a818f23-d88c-4544-9f34-a11d0149a45c admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:12.575724 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 638/1280] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:29:12 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:29:12.576719 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a818f23-d88c-4544-9f34-a11d0149a45c admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:12.602021 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a818f23-d88c-4544-9f34-a11d0149a45c admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:12.610680 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:INSERT=7,keystone:SELECT=93 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:29:12.613416 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a20499-0983-4cc9-b7d7-592d2e73b3b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:12.614102 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a20499-0983-4cc9-b7d7-592d2e73b3b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:12.614417 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a20499-0983-4cc9-b7d7-592d2e73b3b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:12.614741 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a20499-0983-4cc9-b7d7-592d2e73b3b8 admin admin] PATH_INFO: `/v3/projects/24964c0ce73a47ec8fdd270a4e1c770a/users/10315abf3eed42eab4a5a29d07ec07b7/roles/b494fedd00f24f0b84a529e8672a156a` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:12.633573 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a20499-0983-4cc9-b7d7-592d2e73b3b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=24964c0ce73a47ec8fdd270a4e1c770a, user_id=10315abf3eed42eab4a5a29d07ec07b7, role_id=b494fedd00f24f0b84a529e8672a156a)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:12.636462 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a20499-0983-4cc9-b7d7-592d2e73b3b8 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:12.642873 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 643/1281] 10.222.0.18 () {58 vars in 1058 bytes} [Wed Nov 2 07:29:12 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:29:12.644217 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ec247b-7116-4505-b89f-798e5aeb43f1 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:12.644718 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ec247b-7116-4505-b89f-798e5aeb43f1 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:12.644993 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ec247b-7116-4505-b89f-798e5aeb43f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:12.662559 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 639/1282] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Nov 2 07:29:12 2022] PUT /identity/v3/projects/24964c0ce73a47ec8fdd270a4e1c770a/users/10315abf3eed42eab4a5a29d07ec07b7/roles/b494fedd00f24f0b84a529e8672a156a => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:29:12.664550 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf085d94-a918-4b2b-af6f-1a7c5ba6522a None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:12.664823 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf085d94-a918-4b2b-af6f-1a7c5ba6522a None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:12.665167 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf085d94-a918-4b2b-af6f-1a7c5ba6522a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:12.684097 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-99ec247b-7116-4505-b89f-798e5aeb43f1 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:29:12.684895 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-99ec247b-7116-4505-b89f-798e5aeb43f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:12.690988 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:29:12.706235 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-cf085d94-a918-4b2b-af6f-1a7c5ba6522a None None] MFA Rules not processed for user `1bdbe843b0bd4718955df87196c1ec66`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:29:12.715164 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-cf085d94-a918-4b2b-af6f-1a7c5ba6522a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:12.755111 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 644/1283] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:29:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:29:12.756085 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ec247b-7116-4505-b89f-798e5aeb43f1 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:12.773290 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ec247b-7116-4505-b89f-798e5aeb43f1 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:12.779450 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87077b42-b63f-453b-aa09-9d3cb49a0c8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:12.780295 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87077b42-b63f-453b-aa09-9d3cb49a0c8c admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:12.781505 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87077b42-b63f-453b-aa09-9d3cb49a0c8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:12.781505 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87077b42-b63f-453b-aa09-9d3cb49a0c8c admin admin] PATH_INFO: `/v3/projects/52aa71fd3d1b47ffbb4b259d4ad16e40/users/ca1455fb1c024f52b618b06eb8baab13/roles/b494fedd00f24f0b84a529e8672a156a` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:12.799116 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87077b42-b63f-453b-aa09-9d3cb49a0c8c admin admin] RBAC: Authorizing `identity:create_grant(project_id=52aa71fd3d1b47ffbb4b259d4ad16e40, user_id=ca1455fb1c024f52b618b06eb8baab13, role_id=b494fedd00f24f0b84a529e8672a156a)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:12.802792 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87077b42-b63f-453b-aa09-9d3cb49a0c8c admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:12.805989 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 640/1284] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:29:12 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:29:12.807044 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf085d94-a918-4b2b-af6f-1a7c5ba6522a None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:12.826022 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf085d94-a918-4b2b-af6f-1a7c5ba6522a None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:12.829703 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 645/1285] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Nov 2 07:29:12 2022] PUT /identity/v3/projects/52aa71fd3d1b47ffbb4b259d4ad16e40/users/ca1455fb1c024f52b618b06eb8baab13/roles/b494fedd00f24f0b84a529e8672a156a => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:29:12.831041 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87077b42-b63f-453b-aa09-9d3cb49a0c8c admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:12.920245 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b229c5-3758-4c81-9ee3-348cc2d88070 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:12.921041 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b229c5-3758-4c81-9ee3-348cc2d88070 admin admin] REQUEST_METHOD: `PUT` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:12.921313 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b229c5-3758-4c81-9ee3-348cc2d88070 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:12.921588 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b229c5-3758-4c81-9ee3-348cc2d88070 admin admin] PATH_INFO: `/v3/projects/f6181ebe47b443b9b5e57bffd9e08244/users/8a0b861fb7d042c89c1be0101dbbc735/roles/b494fedd00f24f0b84a529e8672a156a` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:12.922773 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87077b42-b63f-453b-aa09-9d3cb49a0c8c admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:12.928158 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a380ca93-9ffb-4f5e-876c-8c5f25a0ee08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:12.929075 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a380ca93-9ffb-4f5e-876c-8c5f25a0ee08 admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:12.929359 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a380ca93-9ffb-4f5e-876c-8c5f25a0ee08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:12.929631 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a380ca93-9ffb-4f5e-876c-8c5f25a0ee08 admin admin] PATH_INFO: `/v3/projects/df168e1504c947169b74f50c1ed4de95/users/2455ebfc92ba4d398a6279d6102a0408/roles/b494fedd00f24f0b84a529e8672a156a` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:12.940329 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b229c5-3758-4c81-9ee3-348cc2d88070 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f6181ebe47b443b9b5e57bffd9e08244, user_id=8a0b861fb7d042c89c1be0101dbbc735, role_id=b494fedd00f24f0b84a529e8672a156a)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:12.943357 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b229c5-3758-4c81-9ee3-348cc2d88070 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:12.951117 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a380ca93-9ffb-4f5e-876c-8c5f25a0ee08 admin admin] RBAC: Authorizing `identity:create_grant(project_id=df168e1504c947169b74f50c1ed4de95, user_id=2455ebfc92ba4d398a6279d6102a0408, role_id=b494fedd00f24f0b84a529e8672a156a)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:12.953947 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a380ca93-9ffb-4f5e-876c-8c5f25a0ee08 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:12.961222 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 641/1286] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Nov 2 07:29:12 2022] PUT /identity/v3/projects/f6181ebe47b443b9b5e57bffd9e08244/users/8a0b861fb7d042c89c1be0101dbbc735/roles/b494fedd00f24f0b84a529e8672a156a => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:29:12.962433 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b229c5-3758-4c81-9ee3-348cc2d88070 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:12.976890 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 646/1287] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Nov 2 07:29:12 2022] PUT /identity/v3/projects/df168e1504c947169b74f50c1ed4de95/users/2455ebfc92ba4d398a6279d6102a0408/roles/b494fedd00f24f0b84a529e8672a156a => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:29:12.979370 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2af2c2e-307f-469c-9e69-0015ff8ce31a None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:12.980747 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2af2c2e-307f-469c-9e69-0015ff8ce31a None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:12.981090 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2af2c2e-307f-469c-9e69-0015ff8ce31a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:13.021219 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-e2af2c2e-307f-469c-9e69-0015ff8ce31a None None] MFA Rules not processed for user `10315abf3eed42eab4a5a29d07ec07b7`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:29:13.022780 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-e2af2c2e-307f-469c-9e69-0015ff8ce31a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:13.050183 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b229c5-3758-4c81-9ee3-348cc2d88070 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:13.054458 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:29:13.079555 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:29:13.131026 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 647/1288] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:29:12 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:29:13.133740 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2af2c2e-307f-469c-9e69-0015ff8ce31a None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:13.136732 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-e2af2c2e-307f-469c-9e69-0015ff8ce31a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:13.159047 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552dd6ea-5d87-454c-a073-d4b14e297804 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e48cba3f804cf08b2477f01ae95179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:13.160327 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552dd6ea-5d87-454c-a073-d4b14e297804 service nova] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:13.160611 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552dd6ea-5d87-454c-a073-d4b14e297804 service nova] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:13.160886 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552dd6ea-5d87-454c-a073-d4b14e297804 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:13.213610 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2af2c2e-307f-469c-9e69-0015ff8ce31a None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:13.220038 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645a85e5-507f-49b8-bedf-24b0a4565cdf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3ec6ecb85f94ef59762877d9ea33ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:13.220038 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645a85e5-507f-49b8-bedf-24b0a4565cdf service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:13.220287 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645a85e5-507f-49b8-bedf-24b0a4565cdf service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:13.220459 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645a85e5-507f-49b8-bedf-24b0a4565cdf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:13.236687 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552dd6ea-5d87-454c-a073-d4b14e297804 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:13.238381 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552dd6ea-5d87-454c-a073-d4b14e297804 service nova] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:13.251926 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 642/1289] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:29:12 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:29:13.253226 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552dd6ea-5d87-454c-a073-d4b14e297804 service nova] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:13.335235 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645a85e5-507f-49b8-bedf-24b0a4565cdf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:13.336313 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645a85e5-507f-49b8-bedf-24b0a4565cdf service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:13.359511 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552dd6ea-5d87-454c-a073-d4b14e297804 service nova] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:13.360466 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 648/1290] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:29:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:29:13.362246 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1f2c1e-5512-4bae-9bc0-2adc8d87c61b None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:13.362556 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1f2c1e-5512-4bae-9bc0-2adc8d87c61b None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:13.362832 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1f2c1e-5512-4bae-9bc0-2adc8d87c61b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:13.373728 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7916c9c-ede2-4e9f-8a9e-cac5c9cddb45 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43bb326fb82e4d2c83fb2eaf91860dc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:13.374370 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7916c9c-ede2-4e9f-8a9e-cac5c9cddb45 service cinder] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:13.377229 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7916c9c-ede2-4e9f-8a9e-cac5c9cddb45 service cinder] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:13.377506 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7916c9c-ede2-4e9f-8a9e-cac5c9cddb45 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:13.440784 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-8f1f2c1e-5512-4bae-9bc0-2adc8d87c61b None None] MFA Rules not processed for user `ca1455fb1c024f52b618b06eb8baab13`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:29:13.442049 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-8f1f2c1e-5512-4bae-9bc0-2adc8d87c61b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:13.535518 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7916c9c-ede2-4e9f-8a9e-cac5c9cddb45 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:13.536999 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7916c9c-ede2-4e9f-8a9e-cac5c9cddb45 service cinder] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:13.561621 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 643/1291] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:29:13 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:29:13.564105 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb95ccb7-0f0d-4d03-9199-8f75a8d95f3b None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:13.564399 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb95ccb7-0f0d-4d03-9199-8f75a8d95f3b None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:13.564675 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb95ccb7-0f0d-4d03-9199-8f75a8d95f3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:13.603801 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 649/1292] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:29:13 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:29:13.606510 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006fcc40-7a9f-4470-9a9d-30e148dd494e None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:13.606830 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006fcc40-7a9f-4470-9a9d-30e148dd494e None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:13.607119 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006fcc40-7a9f-4470-9a9d-30e148dd494e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:13.622124 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:29:13.630384 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-fb95ccb7-0f0d-4d03-9199-8f75a8d95f3b None None] MFA Rules not processed for user `8a0b861fb7d042c89c1be0101dbbc735`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:29:13.633370 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-fb95ccb7-0f0d-4d03-9199-8f75a8d95f3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:13.694734 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:29:13.694734 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-006fcc40-7a9f-4470-9a9d-30e148dd494e None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:29:13.694734 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-006fcc40-7a9f-4470-9a9d-30e148dd494e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:13.697575 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 650/1293] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:29:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:29:13.701186 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006fcc40-7a9f-4470-9a9d-30e148dd494e None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:13.725984 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006fcc40-7a9f-4470-9a9d-30e148dd494e None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:13.736413 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d489bb5-b3da-42a2-bf07-374fff67de30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:13.737268 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d489bb5-b3da-42a2-bf07-374fff67de30 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:13.737655 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d489bb5-b3da-42a2-bf07-374fff67de30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:13.737932 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d489bb5-b3da-42a2-bf07-374fff67de30 admin admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:13.741225 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d489bb5-b3da-42a2-bf07-374fff67de30 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:13.743345 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d489bb5-b3da-42a2-bf07-374fff67de30 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:13.816092 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 651/1294] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Nov 2 07:29:13 2022] POST /identity/v3/projects => generated 367 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:29:13.817752 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d489bb5-b3da-42a2-bf07-374fff67de30 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:13.834665 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 644/1295] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:29:13 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:29:13.840083 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ea470b-7a65-49e6-8a58-f7d2399341dc None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:13.840083 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ea470b-7a65-49e6-8a58-f7d2399341dc None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:13.840083 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ea470b-7a65-49e6-8a58-f7d2399341dc None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:13.850880 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 645/1296] 10.222.0.18 () {54 vars in 912 bytes} [Wed Nov 2 07:29:13 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:29:13.854758 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ea470b-7a65-49e6-8a58-f7d2399341dc None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:13.970508 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d489bb5-b3da-42a2-bf07-374fff67de30 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:13.977026 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f71c12-6036-46b9-b2e4-403f8bbe5bc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:13.979000 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f71c12-6036-46b9-b2e4-403f8bbe5bc2 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:13.980748 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f71c12-6036-46b9-b2e4-403f8bbe5bc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:13.981056 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f71c12-6036-46b9-b2e4-403f8bbe5bc2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:13.982018 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f71c12-6036-46b9-b2e4-403f8bbe5bc2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:13.986452 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f71c12-6036-46b9-b2e4-403f8bbe5bc2 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:14.006844 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ea470b-7a65-49e6-8a58-f7d2399341dc None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:14.030584 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27679de4-1ee5-4957-8f0a-feb4f386dc7e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3ec6ecb85f94ef59762877d9ea33ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:14.031240 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27679de4-1ee5-4957-8f0a-feb4f386dc7e service neutron] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:14.031537 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27679de4-1ee5-4957-8f0a-feb4f386dc7e service neutron] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:14.031782 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27679de4-1ee5-4957-8f0a-feb4f386dc7e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:14.046028 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 652/1297] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Nov 2 07:29:13 2022] POST /identity/v3/projects => generated 405 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:29:14.050323 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f71c12-6036-46b9-b2e4-403f8bbe5bc2 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:14.052220 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27679de4-1ee5-4957-8f0a-feb4f386dc7e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:14.055426 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27679de4-1ee5-4957-8f0a-feb4f386dc7e service neutron] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:14.069927 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 646/1298] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:29:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:29:14.071305 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788c50c9-994b-4d34-83f9-874ab1913f85 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:14.071697 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788c50c9-994b-4d34-83f9-874ab1913f85 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:14.071927 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788c50c9-994b-4d34-83f9-874ab1913f85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:14.111933 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-788c50c9-994b-4d34-83f9-874ab1913f85 None None] MFA Rules not processed for user `69e48cba3f804cf08b2477f01ae95179`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:29:14.113262 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-788c50c9-994b-4d34-83f9-874ab1913f85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:14.192800 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f71c12-6036-46b9-b2e4-403f8bbe5bc2 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:14.203776 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a730d2-9045-4b55-a1a9-b0c5a93ee43e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:14.204489 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a730d2-9045-4b55-a1a9-b0c5a93ee43e admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:14.204842 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a730d2-9045-4b55-a1a9-b0c5a93ee43e admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:14.205210 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a730d2-9045-4b55-a1a9-b0c5a93ee43e admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:14.205905 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a730d2-9045-4b55-a1a9-b0c5a93ee43e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:14.207964 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a730d2-9045-4b55-a1a9-b0c5a93ee43e admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:14.239347 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 653/1299] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Nov 2 07:29:14 2022] POST /identity/v3/users => generated 290 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:29:14.244148 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 647/1300] 10.222.0.18 () {58 vars in 1009 bytes} [Wed Nov 2 07:29:14 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:29:14.245440 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555da9e1-f626-4e68-9e3e-ca0f76e3379e None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:14.247306 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788c50c9-994b-4d34-83f9-874ab1913f85 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:14.247631 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555da9e1-f626-4e68-9e3e-ca0f76e3379e None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:14.248227 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555da9e1-f626-4e68-9e3e-ca0f76e3379e None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:14.250185 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 654/1301] 10.222.0.18 () {54 vars in 961 bytes} [Wed Nov 2 07:29:14 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:29:14.253772 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555da9e1-f626-4e68-9e3e-ca0f76e3379e None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:14.275076 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788c50c9-994b-4d34-83f9-874ab1913f85 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:14.282618 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555da9e1-f626-4e68-9e3e-ca0f76e3379e None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:14.286879 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f711c9e-1dc9-42b8-b37c-47f2e8e4c675 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:14.287537 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f711c9e-1dc9-42b8-b37c-47f2e8e4c675 admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:14.287820 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f711c9e-1dc9-42b8-b37c-47f2e8e4c675 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:14.288099 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f711c9e-1dc9-42b8-b37c-47f2e8e4c675 admin admin] PATH_INFO: `/v3/users` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:14.288768 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f711c9e-1dc9-42b8-b37c-47f2e8e4c675 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:14.290793 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f711c9e-1dc9-42b8-b37c-47f2e8e4c675 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:14.297270 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3134f3-1db5-4aab-8b20-744b5c169bef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:14.297897 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3134f3-1db5-4aab-8b20-744b5c169bef admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:14.298177 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3134f3-1db5-4aab-8b20-744b5c169bef admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:14.298475 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3134f3-1db5-4aab-8b20-744b5c169bef admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:14.299301 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3134f3-1db5-4aab-8b20-744b5c169bef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:14.300704 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3134f3-1db5-4aab-8b20-744b5c169bef admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:14.324370 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 655/1302] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:29:14 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:29:14.325821 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969da927-71c9-412c-b054-54cedf62d9aa None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:14.326197 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969da927-71c9-412c-b054-54cedf62d9aa None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:14.326508 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969da927-71c9-412c-b054-54cedf62d9aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:14.344105 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 648/1303] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Nov 2 07:29:14 2022] POST /identity/v3/users => generated 309 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:29:14.344968 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f711c9e-1dc9-42b8-b37c-47f2e8e4c675 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:14.390284 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-969da927-71c9-412c-b054-54cedf62d9aa None None] MFA Rules not processed for user `69e48cba3f804cf08b2477f01ae95179`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:29:14.391666 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-969da927-71c9-412c-b054-54cedf62d9aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:14.408095 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:29:14.415460 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 656/1304] 10.222.0.18 () {58 vars in 1058 bytes} [Wed Nov 2 07:29:14 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:29:14.416694 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969da927-71c9-412c-b054-54cedf62d9aa None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:14.429430 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969da927-71c9-412c-b054-54cedf62d9aa None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:14.437682 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a835853-b85c-433c-af01-c6a391246d04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:14.438592 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a835853-b85c-433c-af01-c6a391246d04 admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:14.438955 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a835853-b85c-433c-af01-c6a391246d04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:14.439296 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a835853-b85c-433c-af01-c6a391246d04 admin admin] PATH_INFO: `/v3/projects/ba2494e16a1e474180b680c108c16af2/users/2acb239fd3e1465faa60aa9e5701a505/roles/c4bf019fc6c44c71a6c984f6d0bab75f` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:14.459583 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a835853-b85c-433c-af01-c6a391246d04 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba2494e16a1e474180b680c108c16af2, user_id=2acb239fd3e1465faa60aa9e5701a505, role_id=c4bf019fc6c44c71a6c984f6d0bab75f)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:14.464132 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a835853-b85c-433c-af01-c6a391246d04 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:14.492246 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f711c9e-1dc9-42b8-b37c-47f2e8e4c675 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:14.494709 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 657/1305] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Nov 2 07:29:14 2022] PUT /identity/v3/projects/ba2494e16a1e474180b680c108c16af2/users/2acb239fd3e1465faa60aa9e5701a505/roles/c4bf019fc6c44c71a6c984f6d0bab75f => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:29:14.497050 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a835853-b85c-433c-af01-c6a391246d04 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:14.515220 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:29:14.646217 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a835853-b85c-433c-af01-c6a391246d04 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:14.659066 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3d9ee6-40e3-42ba-8d76-c447e497e060 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:14.660089 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3d9ee6-40e3-42ba-8d76-c447e497e060 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:14.661384 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3d9ee6-40e3-42ba-8d76-c447e497e060 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:14.661854 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3d9ee6-40e3-42ba-8d76-c447e497e060 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:14.664144 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3d9ee6-40e3-42ba-8d76-c447e497e060 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:14.670989 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3d9ee6-40e3-42ba-8d76-c447e497e060 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:14.673844 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d230fe99-d938-48fa-aa02-aa47768b667d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43bb326fb82e4d2c83fb2eaf91860dc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:14.674783 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d230fe99-d938-48fa-aa02-aa47768b667d service cinder] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:14.675628 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d230fe99-d938-48fa-aa02-aa47768b667d service cinder] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:14.676110 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d230fe99-d938-48fa-aa02-aa47768b667d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:14.696262 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 658/1306] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:29:14 2022] GET /identity/v3/roles => generated 1420 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:29:14.704044 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3d9ee6-40e3-42ba-8d76-c447e497e060 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:14.820999 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d230fe99-d938-48fa-aa02-aa47768b667d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:14.824615 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d230fe99-d938-48fa-aa02-aa47768b667d service cinder] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:14.840191 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 649/1307] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:29:14 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 495 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:29:14.843025 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d230fe99-d938-48fa-aa02-aa47768b667d service cinder] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:14.858775 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d230fe99-d938-48fa-aa02-aa47768b667d service cinder] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:14.873450 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6920aa-4766-42db-9862-c7c5a9e93fc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:14.874085 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6920aa-4766-42db-9862-c7c5a9e93fc9 admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:14.874623 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6920aa-4766-42db-9862-c7c5a9e93fc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:14.874900 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6920aa-4766-42db-9862-c7c5a9e93fc9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:14.876847 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6920aa-4766-42db-9862-c7c5a9e93fc9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:14.881141 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6920aa-4766-42db-9862-c7c5a9e93fc9 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:14.886884 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3d9ee6-40e3-42ba-8d76-c447e497e060 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:14.904972 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4491064-5eed-4419-8311-3fe65aaa83a7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e48cba3f804cf08b2477f01ae95179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:14.905962 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4491064-5eed-4419-8311-3fe65aaa83a7 service nova] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:14.908132 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4491064-5eed-4419-8311-3fe65aaa83a7 service nova] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:14.908637 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4491064-5eed-4419-8311-3fe65aaa83a7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:14.921321 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 650/1308] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:29:14 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:29:14.922285 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6920aa-4766-42db-9862-c7c5a9e93fc9 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:14.962665 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6920aa-4766-42db-9862-c7c5a9e93fc9 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:14.974997 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb78e48-5006-4923-91f2-813baeb122b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:14.976169 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb78e48-5006-4923-91f2-813baeb122b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:14.976466 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb78e48-5006-4923-91f2-813baeb122b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:14.976845 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb78e48-5006-4923-91f2-813baeb122b2 admin admin] PATH_INFO: `/v3/projects/71a8f4d44f004fc29b58d821f5a0e8b6/users/f2a330527e9c40b8a2c8c7cc1ce090e5/roles/c4bf019fc6c44c71a6c984f6d0bab75f` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:15.017220 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4491064-5eed-4419-8311-3fe65aaa83a7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:15.018871 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4491064-5eed-4419-8311-3fe65aaa83a7 service nova] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:15.024220 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb78e48-5006-4923-91f2-813baeb122b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=71a8f4d44f004fc29b58d821f5a0e8b6, user_id=f2a330527e9c40b8a2c8c7cc1ce090e5, role_id=c4bf019fc6c44c71a6c984f6d0bab75f)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:15.033737 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb78e48-5006-4923-91f2-813baeb122b2 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:15.040782 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 659/1309] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:29:14 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:29:15.041917 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4491064-5eed-4419-8311-3fe65aaa83a7 service nova] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:15.066247 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4491064-5eed-4419-8311-3fe65aaa83a7 service nova] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:15.079682 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a4c550-f96b-4da2-8bbf-4244b4eb754b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:15.080385 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a4c550-f96b-4da2-8bbf-4244b4eb754b admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:15.080624 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a4c550-f96b-4da2-8bbf-4244b4eb754b admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:15.080902 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a4c550-f96b-4da2-8bbf-4244b4eb754b admin admin] PATH_INFO: `/v3/projects/ba2494e16a1e474180b680c108c16af2/users/2acb239fd3e1465faa60aa9e5701a505/roles/c4bf019fc6c44c71a6c984f6d0bab75f` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:15.085550 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 651/1310] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Nov 2 07:29:14 2022] PUT /identity/v3/projects/71a8f4d44f004fc29b58d821f5a0e8b6/users/f2a330527e9c40b8a2c8c7cc1ce090e5/roles/c4bf019fc6c44c71a6c984f6d0bab75f => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:29:15.086331 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a4c550-f96b-4da2-8bbf-4244b4eb754b admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba2494e16a1e474180b680c108c16af2, user_id=2acb239fd3e1465faa60aa9e5701a505, role_id=c4bf019fc6c44c71a6c984f6d0bab75f)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:15.087478 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb78e48-5006-4923-91f2-813baeb122b2 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:15.093968 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a4c550-f96b-4da2-8bbf-4244b4eb754b admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:15.118578 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 660/1311] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Nov 2 07:29:15 2022] PUT /identity/v3/projects/ba2494e16a1e474180b680c108c16af2/users/2acb239fd3e1465faa60aa9e5701a505/roles/c4bf019fc6c44c71a6c984f6d0bab75f => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:29:15.129063 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372f9c0d-2dd0-459f-a131-13fa1d70ec88 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:15.129319 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372f9c0d-2dd0-459f-a131-13fa1d70ec88 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:15.129566 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372f9c0d-2dd0-459f-a131-13fa1d70ec88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:15.175182 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-372f9c0d-2dd0-459f-a131-13fa1d70ec88 None None] MFA Rules not processed for user `2acb239fd3e1465faa60aa9e5701a505`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:29:15.176551 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-372f9c0d-2dd0-459f-a131-13fa1d70ec88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:15.199526 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:29:15.201037 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:29:15.244957 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb78e48-5006-4923-91f2-813baeb122b2 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:15.338149 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 661/1312] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:29:15 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:29:15.349635 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372f9c0d-2dd0-459f-a131-13fa1d70ec88 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:15.389744 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc6b79b-c673-4319-9b47-3ba374123612 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:15.390554 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc6b79b-c673-4319-9b47-3ba374123612 admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:15.390852 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc6b79b-c673-4319-9b47-3ba374123612 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:15.391127 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc6b79b-c673-4319-9b47-3ba374123612 admin admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:15.393654 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc6b79b-c673-4319-9b47-3ba374123612 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:15.395084 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc6b79b-c673-4319-9b47-3ba374123612 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:15.425677 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 652/1313] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:29:15 2022] GET /identity/v3/roles => generated 1420 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 07:29:15.427347 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc6b79b-c673-4319-9b47-3ba374123612 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:15.454673 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc6b79b-c673-4319-9b47-3ba374123612 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:15.469606 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b48c6f7-8dd9-4fe0-8bf3-96052fdb879b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:15.470303 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b48c6f7-8dd9-4fe0-8bf3-96052fdb879b admin admin] REQUEST_METHOD: `PUT` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:15.470609 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b48c6f7-8dd9-4fe0-8bf3-96052fdb879b admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:15.470876 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b48c6f7-8dd9-4fe0-8bf3-96052fdb879b admin admin] PATH_INFO: `/v3/projects/71a8f4d44f004fc29b58d821f5a0e8b6/users/f2a330527e9c40b8a2c8c7cc1ce090e5/roles/c4bf019fc6c44c71a6c984f6d0bab75f` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:15.475479 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b48c6f7-8dd9-4fe0-8bf3-96052fdb879b admin admin] RBAC: Authorizing `identity:create_grant(project_id=71a8f4d44f004fc29b58d821f5a0e8b6, user_id=f2a330527e9c40b8a2c8c7cc1ce090e5, role_id=c4bf019fc6c44c71a6c984f6d0bab75f)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:15.478680 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b48c6f7-8dd9-4fe0-8bf3-96052fdb879b admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:15.512828 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372f9c0d-2dd0-459f-a131-13fa1d70ec88 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:15.523240 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 653/1314] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Nov 2 07:29:15 2022] PUT /identity/v3/projects/71a8f4d44f004fc29b58d821f5a0e8b6/users/f2a330527e9c40b8a2c8c7cc1ce090e5/roles/c4bf019fc6c44c71a6c984f6d0bab75f => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:29:15.534060 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0526b092-b6cd-470b-9205-c23ca8bd9ceb None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:15.534771 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0526b092-b6cd-470b-9205-c23ca8bd9ceb None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:15.535213 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0526b092-b6cd-470b-9205-c23ca8bd9ceb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:15.685691 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-0526b092-b6cd-470b-9205-c23ca8bd9ceb None None] MFA Rules not processed for user `f2a330527e9c40b8a2c8c7cc1ce090e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:29:15.686979 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-0526b092-b6cd-470b-9205-c23ca8bd9ceb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:15.702004 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10b1f77-bf99-43c2-9c29-3cc2f7af5648 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43bb326fb82e4d2c83fb2eaf91860dc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:15.703324 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10b1f77-bf99-43c2-9c29-3cc2f7af5648 service cinder] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:15.705053 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10b1f77-bf99-43c2-9c29-3cc2f7af5648 service cinder] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:15.705552 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10b1f77-bf99-43c2-9c29-3cc2f7af5648 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:15.856010 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10b1f77-bf99-43c2-9c29-3cc2f7af5648 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:15.857466 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10b1f77-bf99-43c2-9c29-3cc2f7af5648 service cinder] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:15.873225 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 662/1315] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:29:15 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 524 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:29:15.875258 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e3dde7-77a5-4b25-9c82-79ca35963eca None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:15.876255 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e3dde7-77a5-4b25-9c82-79ca35963eca None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:15.876255 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e3dde7-77a5-4b25-9c82-79ca35963eca None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:15.877768 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 663/1316] 10.222.0.18 () {54 vars in 912 bytes} [Wed Nov 2 07:29:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:29:15.884468 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23f2b2d-5fd5-4cc3-ad6b-90baf613193f None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:15.884838 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23f2b2d-5fd5-4cc3-ad6b-90baf613193f None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:15.885380 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23f2b2d-5fd5-4cc3-ad6b-90baf613193f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:15.902330 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 654/1317] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:29:15 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 369 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:29:15.940389 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-f23f2b2d-5fd5-4cc3-ad6b-90baf613193f None None] MFA Rules not processed for user `69e48cba3f804cf08b2477f01ae95179`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:29:15.952333 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-f23f2b2d-5fd5-4cc3-ad6b-90baf613193f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:15.979830 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0526b092-b6cd-470b-9205-c23ca8bd9ceb None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:15.993961 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0526b092-b6cd-470b-9205-c23ca8bd9ceb None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:16.003738 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eaf74a1-60ee-402a-ac18-7cdcccc0c5e1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43bb326fb82e4d2c83fb2eaf91860dc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:16.004610 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eaf74a1-60ee-402a-ac18-7cdcccc0c5e1 service cinder] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:16.006063 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eaf74a1-60ee-402a-ac18-7cdcccc0c5e1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:16.006553 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eaf74a1-60ee-402a-ac18-7cdcccc0c5e1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:16.032089 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:29:16.052006 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eaf74a1-60ee-402a-ac18-7cdcccc0c5e1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:16.054165 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eaf74a1-60ee-402a-ac18-7cdcccc0c5e1 service cinder] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:16.086731 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 655/1318] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:29:15 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:29:16.087718 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eaf74a1-60ee-402a-ac18-7cdcccc0c5e1 service cinder] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:16.110772 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eaf74a1-60ee-402a-ac18-7cdcccc0c5e1 service cinder] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:16.122926 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5c80cb-b7bf-4e58-9218-1b7f83904429 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43bb326fb82e4d2c83fb2eaf91860dc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:16.123556 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5c80cb-b7bf-4e58-9218-1b7f83904429 service cinder] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:16.125337 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5c80cb-b7bf-4e58-9218-1b7f83904429 service cinder] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:16.125634 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5c80cb-b7bf-4e58-9218-1b7f83904429 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:16.186536 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 664/1319] 10.222.0.18 () {58 vars in 1009 bytes} [Wed Nov 2 07:29:15 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 303 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:29:16.246847 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:29:16.285711 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5c80cb-b7bf-4e58-9218-1b7f83904429 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:16.287180 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5c80cb-b7bf-4e58-9218-1b7f83904429 service cinder] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:16.315620 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23f2b2d-5fd5-4cc3-ad6b-90baf613193f None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:16.319027 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 656/1320] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:29:16 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:29:16.332549 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23f2b2d-5fd5-4cc3-ad6b-90baf613193f None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:16.343056 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279384c7-923e-4cb4-bd79-3d7d9b1b2f54 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43bb326fb82e4d2c83fb2eaf91860dc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:16.343749 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279384c7-923e-4cb4-bd79-3d7d9b1b2f54 service cinder] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:16.343981 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279384c7-923e-4cb4-bd79-3d7d9b1b2f54 service cinder] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:16.344259 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279384c7-923e-4cb4-bd79-3d7d9b1b2f54 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:16.370292 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279384c7-923e-4cb4-bd79-3d7d9b1b2f54 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:16.372523 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279384c7-923e-4cb4-bd79-3d7d9b1b2f54 service cinder] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:16.401474 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 665/1321] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:29:16 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:29:17.488522 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5c80cb-b7bf-4e58-9218-1b7f83904429 service cinder] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:17.706659 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5c80cb-b7bf-4e58-9218-1b7f83904429 service cinder] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:17.721374 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc73bbf5-32e1-4b76-8676-0a2df6c744e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:17.722134 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc73bbf5-32e1-4b76-8676-0a2df6c744e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:17.722562 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc73bbf5-32e1-4b76-8676-0a2df6c744e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:17.723052 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc73bbf5-32e1-4b76-8676-0a2df6c744e9 admin admin] PATH_INFO: `/v3/users/ca1455fb1c024f52b618b06eb8baab13` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:17.732531 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc73bbf5-32e1-4b76-8676-0a2df6c744e9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ca1455fb1c024f52b618b06eb8baab13)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:17.733977 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: WARNING py.warnings [None req-cc73bbf5-32e1-4b76-8676-0a2df6c744e9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Nov 02 07:29:17.733977 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: warnings.warn(msg) Nov 02 07:29:17.733977 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: Nov 02 07:29:17.735734 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc73bbf5-32e1-4b76-8676-0a2df6c744e9 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:17.883765 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-cc73bbf5-32e1-4b76-8676-0a2df6c744e9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ca1455fb1c024f52b618b06eb8baab13'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:17.913597 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-cc73bbf5-32e1-4b76-8676-0a2df6c744e9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ca1455fb1c024f52b618b06eb8baab13'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:17.914667 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-cc73bbf5-32e1-4b76-8676-0a2df6c744e9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ca1455fb1c024f52b618b06eb8baab13'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:17.946422 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-cc73bbf5-32e1-4b76-8676-0a2df6c744e9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ca1455fb1c024f52b618b06eb8baab13'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:18.008282 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-cc73bbf5-32e1-4b76-8676-0a2df6c744e9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ca1455fb1c024f52b618b06eb8baab13'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:18.026290 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 657/1322] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Nov 2 07:29:17 2022] DELETE /identity/v3/users/ca1455fb1c024f52b618b06eb8baab13 => generated 0 bytes in 538 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:29:18.036082 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139cb4fa-e33d-4725-a9cb-449ee20cb84d None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:18.037034 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139cb4fa-e33d-4725-a9cb-449ee20cb84d None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:18.037754 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139cb4fa-e33d-4725-a9cb-449ee20cb84d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:18.080783 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-139cb4fa-e33d-4725-a9cb-449ee20cb84d None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:29:18.083063 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-139cb4fa-e33d-4725-a9cb-449ee20cb84d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:18.181433 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 666/1323] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:29:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:29:18.192810 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc73bbf5-32e1-4b76-8676-0a2df6c744e9 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:18.194822 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-cc73bbf5-32e1-4b76-8676-0a2df6c744e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:18.442877 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc73bbf5-32e1-4b76-8676-0a2df6c744e9 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:18.447840 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b196ef1-7774-453f-a20a-a3a8ce2559eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3ec6ecb85f94ef59762877d9ea33ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:18.449230 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b196ef1-7774-453f-a20a-a3a8ce2559eb service neutron] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:18.450154 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b196ef1-7774-453f-a20a-a3a8ce2559eb service neutron] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:18.450684 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b196ef1-7774-453f-a20a-a3a8ce2559eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:18.466280 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b196ef1-7774-453f-a20a-a3a8ce2559eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:18.479604 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b196ef1-7774-453f-a20a-a3a8ce2559eb service neutron] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:18.507622 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 658/1324] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:29:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:29:19.243067 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139cb4fa-e33d-4725-a9cb-449ee20cb84d None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:19.244680 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-139cb4fa-e33d-4725-a9cb-449ee20cb84d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:19.314990 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139cb4fa-e33d-4725-a9cb-449ee20cb84d None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:19.321364 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc7dc3a-ffe7-4770-a39b-e1e2b0ee4c6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:19.322012 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc7dc3a-ffe7-4770-a39b-e1e2b0ee4c6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:19.330325 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc7dc3a-ffe7-4770-a39b-e1e2b0ee4c6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:19.330636 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc7dc3a-ffe7-4770-a39b-e1e2b0ee4c6d admin admin] PATH_INFO: `/v3/users/72e9291d458043139a23efef6ff415ea` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:19.333516 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc7dc3a-ffe7-4770-a39b-e1e2b0ee4c6d admin admin] RBAC: Authorizing `identity:delete_user(user_id=72e9291d458043139a23efef6ff415ea)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:19.335763 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc7dc3a-ffe7-4770-a39b-e1e2b0ee4c6d admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:19.507752 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-bfc7dc3a-ffe7-4770-a39b-e1e2b0ee4c6d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '72e9291d458043139a23efef6ff415ea'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:19.525299 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-bfc7dc3a-ffe7-4770-a39b-e1e2b0ee4c6d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '72e9291d458043139a23efef6ff415ea'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:19.526175 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-bfc7dc3a-ffe7-4770-a39b-e1e2b0ee4c6d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '72e9291d458043139a23efef6ff415ea'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:19.547256 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-bfc7dc3a-ffe7-4770-a39b-e1e2b0ee4c6d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '72e9291d458043139a23efef6ff415ea'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:19.552334 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-bfc7dc3a-ffe7-4770-a39b-e1e2b0ee4c6d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '72e9291d458043139a23efef6ff415ea'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:19.604292 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 667/1325] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Nov 2 07:29:19 2022] DELETE /identity/v3/users/72e9291d458043139a23efef6ff415ea => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:29:19.606108 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f65b8ef-b8fd-46bd-98e0-761586532213 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:19.606625 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f65b8ef-b8fd-46bd-98e0-761586532213 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:19.606823 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f65b8ef-b8fd-46bd-98e0-761586532213 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:19.623156 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:29:19.669848 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-9f65b8ef-b8fd-46bd-98e0-761586532213 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:29:19.671232 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-9f65b8ef-b8fd-46bd-98e0-761586532213 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:19.767702 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 659/1326] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:29:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:29:19.796914 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc7dc3a-ffe7-4770-a39b-e1e2b0ee4c6d admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:19.799254 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-bfc7dc3a-ffe7-4770-a39b-e1e2b0ee4c6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:19.853667 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:29:19.974509 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc7dc3a-ffe7-4770-a39b-e1e2b0ee4c6d admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:19.983496 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbe6b6c-8952-4b6e-bd95-e7524d25aebc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3ec6ecb85f94ef59762877d9ea33ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:19.984744 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbe6b6c-8952-4b6e-bd95-e7524d25aebc service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:19.985184 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbe6b6c-8952-4b6e-bd95-e7524d25aebc service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:19.985639 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbe6b6c-8952-4b6e-bd95-e7524d25aebc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:20.002904 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbe6b6c-8952-4b6e-bd95-e7524d25aebc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:20.005519 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbe6b6c-8952-4b6e-bd95-e7524d25aebc service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:20.025680 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 668/1327] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:29:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:29:20.555682 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f65b8ef-b8fd-46bd-98e0-761586532213 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:20.557427 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-9f65b8ef-b8fd-46bd-98e0-761586532213 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:20.593199 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f65b8ef-b8fd-46bd-98e0-761586532213 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:20.609328 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505322a9-34a2-4b67-bf41-0dc8cd91db27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:20.610103 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505322a9-34a2-4b67-bf41-0dc8cd91db27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:20.610261 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505322a9-34a2-4b67-bf41-0dc8cd91db27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:20.610563 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505322a9-34a2-4b67-bf41-0dc8cd91db27 admin admin] PATH_INFO: `/v3/projects/52aa71fd3d1b47ffbb4b259d4ad16e40` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:20.615468 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505322a9-34a2-4b67-bf41-0dc8cd91db27 admin admin] RBAC: Authorizing `identity:delete_project(project_id=52aa71fd3d1b47ffbb4b259d4ad16e40)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:20.628776 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505322a9-34a2-4b67-bf41-0dc8cd91db27 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:20.778402 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-505322a9-34a2-4b67-bf41-0dc8cd91db27 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '52aa71fd3d1b47ffbb4b259d4ad16e40', 'request_id': 'req-505322a9-34a2-4b67-bf41-0dc8cd91db27'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:20.788472 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-505322a9-34a2-4b67-bf41-0dc8cd91db27 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '52aa71fd3d1b47ffbb4b259d4ad16e40', 'request_id': 'req-505322a9-34a2-4b67-bf41-0dc8cd91db27'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:20.805703 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-505322a9-34a2-4b67-bf41-0dc8cd91db27 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '52aa71fd3d1b47ffbb4b259d4ad16e40', 'request_id': 'req-505322a9-34a2-4b67-bf41-0dc8cd91db27'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:20.831761 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-505322a9-34a2-4b67-bf41-0dc8cd91db27 admin admin] The token cache is being invalidate because project 52aa71fd3d1b47ffbb4b259d4ad16e40 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81501) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:29:20.832506 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-505322a9-34a2-4b67-bf41-0dc8cd91db27 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:20.837063 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-505322a9-34a2-4b67-bf41-0dc8cd91db27 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:20.841829 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 660/1328] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Nov 2 07:29:20 2022] DELETE /identity/v3/projects/52aa71fd3d1b47ffbb4b259d4ad16e40 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:29:20.845556 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbe6b6c-8952-4b6e-bd95-e7524d25aebc service neutron] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:20.858904 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbe6b6c-8952-4b6e-bd95-e7524d25aebc service neutron] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:21.047985 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3775a490-911c-408d-8e45-2fd7c0dcc23e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:21.049749 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3775a490-911c-408d-8e45-2fd7c0dcc23e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:21.050178 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3775a490-911c-408d-8e45-2fd7c0dcc23e admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:21.050640 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3775a490-911c-408d-8e45-2fd7c0dcc23e admin admin] PATH_INFO: `/v3/users/f2a330527e9c40b8a2c8c7cc1ce090e5` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:21.058424 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3775a490-911c-408d-8e45-2fd7c0dcc23e admin admin] RBAC: Authorizing `identity:delete_user(user_id=f2a330527e9c40b8a2c8c7cc1ce090e5)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:21.061869 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3775a490-911c-408d-8e45-2fd7c0dcc23e admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:21.169345 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-3775a490-911c-408d-8e45-2fd7c0dcc23e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f2a330527e9c40b8a2c8c7cc1ce090e5'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:21.192027 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-3775a490-911c-408d-8e45-2fd7c0dcc23e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f2a330527e9c40b8a2c8c7cc1ce090e5'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:21.192213 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-3775a490-911c-408d-8e45-2fd7c0dcc23e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f2a330527e9c40b8a2c8c7cc1ce090e5'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:21.207583 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-3775a490-911c-408d-8e45-2fd7c0dcc23e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f2a330527e9c40b8a2c8c7cc1ce090e5'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:21.208756 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-3775a490-911c-408d-8e45-2fd7c0dcc23e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f2a330527e9c40b8a2c8c7cc1ce090e5'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:21.258626 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505322a9-34a2-4b67-bf41-0dc8cd91db27 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:21.264039 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-505322a9-34a2-4b67-bf41-0dc8cd91db27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:21.265545 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 669/1329] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Nov 2 07:29:20 2022] DELETE /identity/v3/users/f2a330527e9c40b8a2c8c7cc1ce090e5 => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:29:21.288260 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505322a9-34a2-4b67-bf41-0dc8cd91db27 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:21.413305 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3da555-6e11-425f-b423-b7ce482f812b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:21.414871 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3da555-6e11-425f-b423-b7ce482f812b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:21.416466 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3da555-6e11-425f-b423-b7ce482f812b admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:21.416923 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3da555-6e11-425f-b423-b7ce482f812b admin admin] PATH_INFO: `/v3/users/10315abf3eed42eab4a5a29d07ec07b7` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:21.419715 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3da555-6e11-425f-b423-b7ce482f812b admin admin] RBAC: Authorizing `identity:delete_user(user_id=10315abf3eed42eab4a5a29d07ec07b7)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:21.421884 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3da555-6e11-425f-b423-b7ce482f812b admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:21.477378 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:29:21.503243 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-3d3da555-6e11-425f-b423-b7ce482f812b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '10315abf3eed42eab4a5a29d07ec07b7'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:21.514491 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-3d3da555-6e11-425f-b423-b7ce482f812b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '10315abf3eed42eab4a5a29d07ec07b7'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:21.515485 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-3d3da555-6e11-425f-b423-b7ce482f812b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '10315abf3eed42eab4a5a29d07ec07b7'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:21.529229 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-3d3da555-6e11-425f-b423-b7ce482f812b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '10315abf3eed42eab4a5a29d07ec07b7'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:21.570157 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-3d3da555-6e11-425f-b423-b7ce482f812b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '10315abf3eed42eab4a5a29d07ec07b7'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:21.584207 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 661/1330] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Nov 2 07:29:21 2022] DELETE /identity/v3/users/10315abf3eed42eab4a5a29d07ec07b7 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:29:21.586682 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba27423-1594-4031-87f1-bae36bb1afc3 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:21.586917 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba27423-1594-4031-87f1-bae36bb1afc3 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:21.587173 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba27423-1594-4031-87f1-bae36bb1afc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:21.622611 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-eba27423-1594-4031-87f1-bae36bb1afc3 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:29:21.624763 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-eba27423-1594-4031-87f1-bae36bb1afc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:21.716970 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:29:21.760685 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3da555-6e11-425f-b423-b7ce482f812b admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:21.762786 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-3d3da555-6e11-425f-b423-b7ce482f812b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:21.776519 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 670/1331] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:29:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:29:21.790782 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3da555-6e11-425f-b423-b7ce482f812b admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:21.805713 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c93cc84-5153-47ee-8efc-d862ff22b65e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:21.806635 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c93cc84-5153-47ee-8efc-d862ff22b65e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:21.807126 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c93cc84-5153-47ee-8efc-d862ff22b65e admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:21.808002 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c93cc84-5153-47ee-8efc-d862ff22b65e admin admin] PATH_INFO: `/v3/projects/948d495ca7dc4cc4a2b8cb2e2d621324` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:21.810177 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c93cc84-5153-47ee-8efc-d862ff22b65e admin admin] RBAC: Authorizing `identity:delete_project(project_id=948d495ca7dc4cc4a2b8cb2e2d621324)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:21.812301 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c93cc84-5153-47ee-8efc-d862ff22b65e admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:21.828201 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba27423-1594-4031-87f1-bae36bb1afc3 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:21.832863 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-eba27423-1594-4031-87f1-bae36bb1afc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:21.981247 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-7c93cc84-5153-47ee-8efc-d862ff22b65e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '948d495ca7dc4cc4a2b8cb2e2d621324', 'request_id': 'req-7c93cc84-5153-47ee-8efc-d862ff22b65e'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:22.006552 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-7c93cc84-5153-47ee-8efc-d862ff22b65e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '948d495ca7dc4cc4a2b8cb2e2d621324', 'request_id': 'req-7c93cc84-5153-47ee-8efc-d862ff22b65e'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:22.027385 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-7c93cc84-5153-47ee-8efc-d862ff22b65e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '948d495ca7dc4cc4a2b8cb2e2d621324', 'request_id': 'req-7c93cc84-5153-47ee-8efc-d862ff22b65e'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:22.037512 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba27423-1594-4031-87f1-bae36bb1afc3 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:22.050816 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-7c93cc84-5153-47ee-8efc-d862ff22b65e admin admin] The token cache is being invalidate because project 948d495ca7dc4cc4a2b8cb2e2d621324 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81501) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:29:22.051103 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-7c93cc84-5153-47ee-8efc-d862ff22b65e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:22.054084 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-7c93cc84-5153-47ee-8efc-d862ff22b65e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:22.063500 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 662/1332] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Nov 2 07:29:21 2022] DELETE /identity/v3/projects/948d495ca7dc4cc4a2b8cb2e2d621324 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 07:29:22.064459 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4512920-fe5e-4e5d-8a30-5986d56c2b0e None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:22.066465 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4512920-fe5e-4e5d-8a30-5986d56c2b0e None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:22.066992 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4512920-fe5e-4e5d-8a30-5986d56c2b0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:22.117307 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-d4512920-fe5e-4e5d-8a30-5986d56c2b0e None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:29:22.118554 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-d4512920-fe5e-4e5d-8a30-5986d56c2b0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:22.197437 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3eece2-dab9-47c4-9b81-9c61e193253e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3ec6ecb85f94ef59762877d9ea33ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:22.200540 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3eece2-dab9-47c4-9b81-9c61e193253e service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:22.201110 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3eece2-dab9-47c4-9b81-9c61e193253e service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:22.201687 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3eece2-dab9-47c4-9b81-9c61e193253e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:22.207158 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-4c3eece2-dab9-47c4-9b81-9c61e193253e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:22.359961 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 663/1333] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:29:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 297 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:29:22.369312 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b964f5-0a70-4782-9ac9-1468cec5c5fc None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:22.369617 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b964f5-0a70-4782-9ac9-1468cec5c5fc None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:22.371721 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b964f5-0a70-4782-9ac9-1468cec5c5fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:22.437358 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3eece2-dab9-47c4-9b81-9c61e193253e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:22.439989 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3eece2-dab9-47c4-9b81-9c61e193253e service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:22.455173 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 671/1334] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:29:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 645 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:29:22.462882 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-d7b964f5-0a70-4782-9ac9-1468cec5c5fc None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:29:22.473386 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-d7b964f5-0a70-4782-9ac9-1468cec5c5fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:22.496611 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 664/1335] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:29:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:29:22.499411 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3eece2-dab9-47c4-9b81-9c61e193253e service neutron] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:22.535547 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3eece2-dab9-47c4-9b81-9c61e193253e service neutron] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:22.550608 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badd953e-c281-4b59-b082-743b18b2e6b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:22.551239 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badd953e-c281-4b59-b082-743b18b2e6b0 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:22.551518 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badd953e-c281-4b59-b082-743b18b2e6b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:22.551790 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badd953e-c281-4b59-b082-743b18b2e6b0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:22.552448 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badd953e-c281-4b59-b082-743b18b2e6b0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:29:22.552747 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badd953e-c281-4b59-b082-743b18b2e6b0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:22.554689 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badd953e-c281-4b59-b082-743b18b2e6b0 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:22.585822 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 672/1336] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Nov 2 07:29:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:29:22.594170 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b964f5-0a70-4782-9ac9-1468cec5c5fc None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:22.606039 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b964f5-0a70-4782-9ac9-1468cec5c5fc None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:22.622455 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2ea0c9-6cd7-4bf5-b5bb-82646e267d1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:22.622455 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2ea0c9-6cd7-4bf5-b5bb-82646e267d1c admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:22.622455 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2ea0c9-6cd7-4bf5-b5bb-82646e267d1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:22.622455 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2ea0c9-6cd7-4bf5-b5bb-82646e267d1c admin admin] PATH_INFO: `/v3/projects` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:22.628011 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2ea0c9-6cd7-4bf5-b5bb-82646e267d1c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:22.631375 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2ea0c9-6cd7-4bf5-b5bb-82646e267d1c admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:22.727464 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badd953e-c281-4b59-b082-743b18b2e6b0 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:22.728946 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 665/1337] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Nov 2 07:29:22 2022] POST /identity/v3/projects => generated 385 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:29:22.741438 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badd953e-c281-4b59-b082-743b18b2e6b0 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:22.846627 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:29:22.856442 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2ea0c9-6cd7-4bf5-b5bb-82646e267d1c admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:22.859569 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-5a2ea0c9-6cd7-4bf5-b5bb-82646e267d1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:22.865253 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:29:22.920419 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9f8de7-7096-449b-be4f-88e9c955ebbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:22.921050 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9f8de7-7096-449b-be4f-88e9c955ebbd admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:22.921331 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9f8de7-7096-449b-be4f-88e9c955ebbd admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:22.921596 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9f8de7-7096-449b-be4f-88e9c955ebbd admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:22.922249 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9f8de7-7096-449b-be4f-88e9c955ebbd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:22.924240 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9f8de7-7096-449b-be4f-88e9c955ebbd admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:22.991654 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 673/1338] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Nov 2 07:29:22 2022] POST /identity/v3/users => generated 293 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 07:29:22.992976 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9f8de7-7096-449b-be4f-88e9c955ebbd admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:23.005166 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9f8de7-7096-449b-be4f-88e9c955ebbd admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:23.010458 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4bc478-398c-49f9-a15c-4bec29204b9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:23.012089 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4bc478-398c-49f9-a15c-4bec29204b9a admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:23.012642 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4bc478-398c-49f9-a15c-4bec29204b9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:23.013133 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4bc478-398c-49f9-a15c-4bec29204b9a admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:23.014002 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4bc478-398c-49f9-a15c-4bec29204b9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:23.016520 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4bc478-398c-49f9-a15c-4bec29204b9a admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:23.030305 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2ea0c9-6cd7-4bf5-b5bb-82646e267d1c admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:23.035393 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 674/1339] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:29:22 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:29:23.036649 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4bc478-398c-49f9-a15c-4bec29204b9a admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:23.047442 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b555c122-b47b-43c6-ae1b-55f3eadaa5d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:23.048257 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b555c122-b47b-43c6-ae1b-55f3eadaa5d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:23.048792 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b555c122-b47b-43c6-ae1b-55f3eadaa5d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:23.049251 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b555c122-b47b-43c6-ae1b-55f3eadaa5d8 admin admin] PATH_INFO: `/v3/projects/71a8f4d44f004fc29b58d821f5a0e8b6` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:23.051747 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4bc478-398c-49f9-a15c-4bec29204b9a admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:23.052741 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b555c122-b47b-43c6-ae1b-55f3eadaa5d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=71a8f4d44f004fc29b58d821f5a0e8b6)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:23.056050 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b555c122-b47b-43c6-ae1b-55f3eadaa5d8 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:23.062251 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a14bb96-d27c-4f9a-8199-3b9b92e86235 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:23.063031 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a14bb96-d27c-4f9a-8199-3b9b92e86235 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:23.063420 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a14bb96-d27c-4f9a-8199-3b9b92e86235 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:23.063800 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a14bb96-d27c-4f9a-8199-3b9b92e86235 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:23.064586 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a14bb96-d27c-4f9a-8199-3b9b92e86235 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:23.068530 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a14bb96-d27c-4f9a-8199-3b9b92e86235 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:23.093393 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 675/1340] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:29:23 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:29:23.098544 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a14bb96-d27c-4f9a-8199-3b9b92e86235 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:23.125966 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a14bb96-d27c-4f9a-8199-3b9b92e86235 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:23.139994 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-b555c122-b47b-43c6-ae1b-55f3eadaa5d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '71a8f4d44f004fc29b58d821f5a0e8b6', 'request_id': 'req-b555c122-b47b-43c6-ae1b-55f3eadaa5d8'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:23.156895 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-b555c122-b47b-43c6-ae1b-55f3eadaa5d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '71a8f4d44f004fc29b58d821f5a0e8b6', 'request_id': 'req-b555c122-b47b-43c6-ae1b-55f3eadaa5d8'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:23.176364 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-b555c122-b47b-43c6-ae1b-55f3eadaa5d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '71a8f4d44f004fc29b58d821f5a0e8b6', 'request_id': 'req-b555c122-b47b-43c6-ae1b-55f3eadaa5d8'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:23.193059 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-b555c122-b47b-43c6-ae1b-55f3eadaa5d8 admin admin] The token cache is being invalidate because project 71a8f4d44f004fc29b58d821f5a0e8b6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81501) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:29:23.195374 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-b555c122-b47b-43c6-ae1b-55f3eadaa5d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:23.197689 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-b555c122-b47b-43c6-ae1b-55f3eadaa5d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:23.200109 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 666/1341] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Nov 2 07:29:22 2022] DELETE /identity/v3/projects/71a8f4d44f004fc29b58d821f5a0e8b6 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:29:23.206140 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a0bb13-9cd5-4a8f-a4e1-aa9e5ee29291 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:23.208042 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a0bb13-9cd5-4a8f-a4e1-aa9e5ee29291 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:23.208396 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a0bb13-9cd5-4a8f-a4e1-aa9e5ee29291 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:23.249507 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-c9a0bb13-9cd5-4a8f-a4e1-aa9e5ee29291 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:29:23.251504 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-c9a0bb13-9cd5-4a8f-a4e1-aa9e5ee29291 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:23.259701 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a34a92f-dd58-438f-9a9c-2fff897e5fe6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:23.260613 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a34a92f-dd58-438f-9a9c-2fff897e5fe6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:23.260998 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a34a92f-dd58-438f-9a9c-2fff897e5fe6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:23.262675 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a34a92f-dd58-438f-9a9c-2fff897e5fe6 admin admin] PATH_INFO: `/v3/projects/2bd29447b6134170804715ea2125fa7a/users/fee48a9736c94a76930f99d4b7b11646/roles/b494fedd00f24f0b84a529e8672a156a` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:23.283648 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 667/1342] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:29:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:29:23.285257 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3b6562-15ac-4078-8e4a-633f9e965552 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:23.286196 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3b6562-15ac-4078-8e4a-633f9e965552 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:23.288199 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3b6562-15ac-4078-8e4a-633f9e965552 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:23.314238 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a34a92f-dd58-438f-9a9c-2fff897e5fe6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2bd29447b6134170804715ea2125fa7a, user_id=fee48a9736c94a76930f99d4b7b11646, role_id=b494fedd00f24f0b84a529e8672a156a)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:23.317458 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a34a92f-dd58-438f-9a9c-2fff897e5fe6 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:23.362864 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 676/1343] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Nov 2 07:29:23 2022] PUT /identity/v3/projects/2bd29447b6134170804715ea2125fa7a/users/fee48a9736c94a76930f99d4b7b11646/roles/b494fedd00f24f0b84a529e8672a156a => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:29:23.367168 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8013cb2c-1d60-4051-bee4-c1fe89cc39b1 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:23.367168 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8013cb2c-1d60-4051-bee4-c1fe89cc39b1 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:23.367168 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8013cb2c-1d60-4051-bee4-c1fe89cc39b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:23.369320 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-7c3b6562-15ac-4078-8e4a-633f9e965552 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:29:23.372365 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-7c3b6562-15ac-4078-8e4a-633f9e965552 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:23.475668 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-8013cb2c-1d60-4051-bee4-c1fe89cc39b1 None None] MFA Rules not processed for user `fee48a9736c94a76930f99d4b7b11646`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:29:23.477150 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-8013cb2c-1d60-4051-bee4-c1fe89cc39b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:23.485439 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 668/1344] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:29:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:29:23.491605 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3b6562-15ac-4078-8e4a-633f9e965552 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:23.515317 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3b6562-15ac-4078-8e4a-633f9e965552 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:23.546918 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e8c9c4-7079-49a9-963d-04c8d053cf5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:23.547887 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e8c9c4-7079-49a9-963d-04c8d053cf5f admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:23.549413 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e8c9c4-7079-49a9-963d-04c8d053cf5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:23.549920 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e8c9c4-7079-49a9-963d-04c8d053cf5f admin admin] PATH_INFO: `/v3/domains` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:23.554401 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e8c9c4-7079-49a9-963d-04c8d053cf5f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:29:23.555444 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e8c9c4-7079-49a9-963d-04c8d053cf5f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:23.562236 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e8c9c4-7079-49a9-963d-04c8d053cf5f admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:23.586953 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 669/1345] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Nov 2 07:29:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:29:23.593741 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e8c9c4-7079-49a9-963d-04c8d053cf5f admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:23.629398 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e8c9c4-7079-49a9-963d-04c8d053cf5f admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:23.657131 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 677/1346] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:29:23 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 294 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:29:23.661696 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8013cb2c-1d60-4051-bee4-c1fe89cc39b1 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:23.663665 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-8013cb2c-1d60-4051-bee4-c1fe89cc39b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:23.664434 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b88c355-5717-4a5c-af0e-b59cc02627a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:23.666100 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b88c355-5717-4a5c-af0e-b59cc02627a3 admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:23.666561 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b88c355-5717-4a5c-af0e-b59cc02627a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:23.667407 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b88c355-5717-4a5c-af0e-b59cc02627a3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:23.669615 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b88c355-5717-4a5c-af0e-b59cc02627a3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:23.672548 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b88c355-5717-4a5c-af0e-b59cc02627a3 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:23.717030 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8013cb2c-1d60-4051-bee4-c1fe89cc39b1 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:23.739352 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:29:23.747008 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0716f2-4d6d-4790-bc68-8fb48de0e602 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:23.747449 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0716f2-4d6d-4790-bc68-8fb48de0e602 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:23.747734 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0716f2-4d6d-4790-bc68-8fb48de0e602 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:23.748008 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0716f2-4d6d-4790-bc68-8fb48de0e602 admin admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:23.748738 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0716f2-4d6d-4790-bc68-8fb48de0e602 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:23.759042 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0716f2-4d6d-4790-bc68-8fb48de0e602 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:23.775955 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:29:23.799456 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 670/1347] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Nov 2 07:29:23 2022] POST /identity/v3/projects => generated 403 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:29:23.808322 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b88c355-5717-4a5c-af0e-b59cc02627a3 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:23.862473 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 678/1348] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Nov 2 07:29:23 2022] POST /identity/v3/projects => generated 385 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:29:23.867281 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0716f2-4d6d-4790-bc68-8fb48de0e602 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:24.029924 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b88c355-5717-4a5c-af0e-b59cc02627a3 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:24.065325 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0716f2-4d6d-4790-bc68-8fb48de0e602 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:24.083212 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c45d6ce-ddf6-4727-9b39-6873fe52ec00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:24.086052 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c45d6ce-ddf6-4727-9b39-6873fe52ec00 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:24.086369 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c45d6ce-ddf6-4727-9b39-6873fe52ec00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:24.086799 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c45d6ce-ddf6-4727-9b39-6873fe52ec00 admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:24.091236 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c45d6ce-ddf6-4727-9b39-6873fe52ec00 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:24.096042 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c45d6ce-ddf6-4727-9b39-6873fe52ec00 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:24.153611 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 679/1349] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Nov 2 07:29:23 2022] POST /identity/v3/users => generated 299 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:29:24.157307 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c45d6ce-ddf6-4727-9b39-6873fe52ec00 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:24.180448 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c45d6ce-ddf6-4727-9b39-6873fe52ec00 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:24.193270 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83f39ff-496a-4e1e-a0d2-5587541b0e8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:24.194097 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83f39ff-496a-4e1e-a0d2-5587541b0e8b admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:24.194690 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83f39ff-496a-4e1e-a0d2-5587541b0e8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:24.195084 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83f39ff-496a-4e1e-a0d2-5587541b0e8b admin admin] PATH_INFO: `/v3/users` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:24.195270 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab88857-4e80-454b-884f-8ee3378b804d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:24.195901 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab88857-4e80-454b-884f-8ee3378b804d admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:24.196180 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab88857-4e80-454b-884f-8ee3378b804d admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:24.196449 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83f39ff-496a-4e1e-a0d2-5587541b0e8b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:24.196586 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab88857-4e80-454b-884f-8ee3378b804d admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:24.198054 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab88857-4e80-454b-884f-8ee3378b804d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:24.200076 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83f39ff-496a-4e1e-a0d2-5587541b0e8b admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:24.205413 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab88857-4e80-454b-884f-8ee3378b804d admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:24.233064 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 680/1350] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:29:24 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 07:29:24.236127 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e60be6c-c7ac-416e-a974-62de0e014404 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:24.237063 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e60be6c-c7ac-416e-a974-62de0e014404 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:24.237063 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e60be6c-c7ac-416e-a974-62de0e014404 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:24.273413 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 671/1351] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Nov 2 07:29:23 2022] POST /identity/v3/users => generated 302 bytes in 469 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:29:24.274509 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83f39ff-496a-4e1e-a0d2-5587541b0e8b admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:24.299345 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83f39ff-496a-4e1e-a0d2-5587541b0e8b admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:24.313605 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dff71ce-8322-475f-a322-0ab0636d4742 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:24.314353 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dff71ce-8322-475f-a322-0ab0636d4742 admin admin] REQUEST_METHOD: `PUT` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:24.314805 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dff71ce-8322-475f-a322-0ab0636d4742 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:24.315087 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dff71ce-8322-475f-a322-0ab0636d4742 admin admin] PATH_INFO: `/v3/projects/44c8892cee764351b8eaa1eb4187cdc3/users/e204cd785f3045ae815cd625e449a7e8/roles/c4bf019fc6c44c71a6c984f6d0bab75f` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:24.337005 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-1e60be6c-c7ac-416e-a974-62de0e014404 None None] MFA Rules not processed for user `0e571b0f4d6f4fafb71939f501b8deec`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:29:24.341027 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-1e60be6c-c7ac-416e-a974-62de0e014404 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:24.350681 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dff71ce-8322-475f-a322-0ab0636d4742 admin admin] RBAC: Authorizing `identity:create_grant(project_id=44c8892cee764351b8eaa1eb4187cdc3, user_id=e204cd785f3045ae815cd625e449a7e8, role_id=c4bf019fc6c44c71a6c984f6d0bab75f)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:24.353911 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dff71ce-8322-475f-a322-0ab0636d4742 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:24.382055 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 672/1352] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Nov 2 07:29:24 2022] PUT /identity/v3/projects/44c8892cee764351b8eaa1eb4187cdc3/users/e204cd785f3045ae815cd625e449a7e8/roles/c4bf019fc6c44c71a6c984f6d0bab75f => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:29:24.382953 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dff71ce-8322-475f-a322-0ab0636d4742 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:24.586432 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dff71ce-8322-475f-a322-0ab0636d4742 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:24.587827 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 681/1353] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:29:24 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 354 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:29:24.589760 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e60be6c-c7ac-416e-a974-62de0e014404 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:24.595596 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c94ba08-6800-494b-a138-b9aaa592049a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:24.597547 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c94ba08-6800-494b-a138-b9aaa592049a admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:24.602020 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c94ba08-6800-494b-a138-b9aaa592049a admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:24.602237 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c94ba08-6800-494b-a138-b9aaa592049a admin admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:24.604156 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c94ba08-6800-494b-a138-b9aaa592049a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:24.623486 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c94ba08-6800-494b-a138-b9aaa592049a admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:24.633532 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e60be6c-c7ac-416e-a974-62de0e014404 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:24.640479 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:29:24.645946 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f362ff4e-3068-4627-aff8-b7c0cb370af0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:24.646638 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f362ff4e-3068-4627-aff8-b7c0cb370af0 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:24.647144 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f362ff4e-3068-4627-aff8-b7c0cb370af0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:24.647426 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f362ff4e-3068-4627-aff8-b7c0cb370af0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:24.648148 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f362ff4e-3068-4627-aff8-b7c0cb370af0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:24.654180 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f362ff4e-3068-4627-aff8-b7c0cb370af0 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:24.662742 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 673/1354] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:29:24 2022] GET /identity/v3/roles => generated 1420 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:29:24.663626 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c94ba08-6800-494b-a138-b9aaa592049a admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:24.681784 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-9c94ba08-6800-494b-a138-b9aaa592049a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:24.686384 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 682/1355] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:29:24 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:29:24.689722 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da8baae-f544-4514-86d8-bfeaeee41bba None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:24.690027 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da8baae-f544-4514-86d8-bfeaeee41bba None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:24.690308 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da8baae-f544-4514-86d8-bfeaeee41bba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:24.732193 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c94ba08-6800-494b-a138-b9aaa592049a admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:24.753361 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-7da8baae-f544-4514-86d8-bfeaeee41bba None None] MFA Rules not processed for user `0e571b0f4d6f4fafb71939f501b8deec`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:29:24.757103 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-7da8baae-f544-4514-86d8-bfeaeee41bba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:24.772290 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656b4ca9-33f5-46e9-9295-d1d6f1f3e48e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:24.772290 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656b4ca9-33f5-46e9-9295-d1d6f1f3e48e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:24.772290 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656b4ca9-33f5-46e9-9295-d1d6f1f3e48e admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:24.772290 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656b4ca9-33f5-46e9-9295-d1d6f1f3e48e admin admin] PATH_INFO: `/v3/projects/24964c0ce73a47ec8fdd270a4e1c770a` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:24.776497 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656b4ca9-33f5-46e9-9295-d1d6f1f3e48e admin admin] RBAC: Authorizing `identity:delete_project(project_id=24964c0ce73a47ec8fdd270a4e1c770a)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:24.786125 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656b4ca9-33f5-46e9-9295-d1d6f1f3e48e admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:24.858962 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:29:24.906884 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 683/1356] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:29:24 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:29:24.906884 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da8baae-f544-4514-86d8-bfeaeee41bba None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:24.932594 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da8baae-f544-4514-86d8-bfeaeee41bba None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:24.970800 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-656b4ca9-33f5-46e9-9295-d1d6f1f3e48e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '24964c0ce73a47ec8fdd270a4e1c770a', 'request_id': 'req-656b4ca9-33f5-46e9-9295-d1d6f1f3e48e'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:24.984117 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-656b4ca9-33f5-46e9-9295-d1d6f1f3e48e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '24964c0ce73a47ec8fdd270a4e1c770a', 'request_id': 'req-656b4ca9-33f5-46e9-9295-d1d6f1f3e48e'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:25.001332 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-656b4ca9-33f5-46e9-9295-d1d6f1f3e48e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '24964c0ce73a47ec8fdd270a4e1c770a', 'request_id': 'req-656b4ca9-33f5-46e9-9295-d1d6f1f3e48e'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:25.017908 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-656b4ca9-33f5-46e9-9295-d1d6f1f3e48e admin admin] The token cache is being invalidate because project 24964c0ce73a47ec8fdd270a4e1c770a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81501) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:29:25.018681 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-656b4ca9-33f5-46e9-9295-d1d6f1f3e48e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:25.019572 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-656b4ca9-33f5-46e9-9295-d1d6f1f3e48e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:25.021371 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 674/1357] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Nov 2 07:29:24 2022] DELETE /identity/v3/projects/24964c0ce73a47ec8fdd270a4e1c770a => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:29:25.022782 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656b4ca9-33f5-46e9-9295-d1d6f1f3e48e admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:25.025063 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-656b4ca9-33f5-46e9-9295-d1d6f1f3e48e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:25.055905 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fab63f5-d0ec-47d9-ac2c-7986a7bb48cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:25.056514 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fab63f5-d0ec-47d9-ac2c-7986a7bb48cc admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:25.056789 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fab63f5-d0ec-47d9-ac2c-7986a7bb48cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:25.057057 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fab63f5-d0ec-47d9-ac2c-7986a7bb48cc admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:25.057700 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fab63f5-d0ec-47d9-ac2c-7986a7bb48cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:25.059432 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fab63f5-d0ec-47d9-ac2c-7986a7bb48cc admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:25.071069 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 684/1358] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:29:24 2022] GET /identity/v3/roles => generated 1420 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:29:25.071830 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fab63f5-d0ec-47d9-ac2c-7986a7bb48cc admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:25.076529 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-9fab63f5-d0ec-47d9-ac2c-7986a7bb48cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:25.179749 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656b4ca9-33f5-46e9-9295-d1d6f1f3e48e admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:25.187109 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4096c52-f15a-424d-96a8-f1950d81e2fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:25.187930 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4096c52-f15a-424d-96a8-f1950d81e2fe admin admin] REQUEST_METHOD: `PUT` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:25.188269 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4096c52-f15a-424d-96a8-f1950d81e2fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:25.188748 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4096c52-f15a-424d-96a8-f1950d81e2fe admin admin] PATH_INFO: `/v3/projects/44c8892cee764351b8eaa1eb4187cdc3/users/e204cd785f3045ae815cd625e449a7e8/roles/c4bf019fc6c44c71a6c984f6d0bab75f` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:25.191913 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4096c52-f15a-424d-96a8-f1950d81e2fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=44c8892cee764351b8eaa1eb4187cdc3, user_id=e204cd785f3045ae815cd625e449a7e8, role_id=c4bf019fc6c44c71a6c984f6d0bab75f)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:25.197407 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4096c52-f15a-424d-96a8-f1950d81e2fe admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:25.224539 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 675/1359] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Nov 2 07:29:25 2022] PUT /identity/v3/projects/44c8892cee764351b8eaa1eb4187cdc3/users/e204cd785f3045ae815cd625e449a7e8/roles/c4bf019fc6c44c71a6c984f6d0bab75f => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:29:25.227818 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4096c52-f15a-424d-96a8-f1950d81e2fe admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:25.234347 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-f4096c52-f15a-424d-96a8-f1950d81e2fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:25.240175 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fab63f5-d0ec-47d9-ac2c-7986a7bb48cc admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:25.405834 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1816e1db-b603-4d0e-abfb-1f040ca08205 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e48cba3f804cf08b2477f01ae95179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:25.406641 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1816e1db-b603-4d0e-abfb-1f040ca08205 service nova] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:25.406922 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1816e1db-b603-4d0e-abfb-1f040ca08205 service nova] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:25.407191 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1816e1db-b603-4d0e-abfb-1f040ca08205 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:25.415064 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4096c52-f15a-424d-96a8-f1950d81e2fe admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:25.415276 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-1816e1db-b603-4d0e-abfb-1f040ca08205 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:25.421755 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e7860f-dbb2-4a7a-9b04-decd149ef754 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:25.424969 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e7860f-dbb2-4a7a-9b04-decd149ef754 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:25.425274 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e7860f-dbb2-4a7a-9b04-decd149ef754 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:25.425631 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e7860f-dbb2-4a7a-9b04-decd149ef754 admin admin] PATH_INFO: `/v3/users/8a0b861fb7d042c89c1be0101dbbc735` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:25.427778 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e7860f-dbb2-4a7a-9b04-decd149ef754 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a0b861fb7d042c89c1be0101dbbc735)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:25.431454 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e7860f-dbb2-4a7a-9b04-decd149ef754 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:25.496347 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:29:25.512310 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:29:25.593448 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-13e7860f-dbb2-4a7a-9b04-decd149ef754 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a0b861fb7d042c89c1be0101dbbc735'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:25.613770 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-13e7860f-dbb2-4a7a-9b04-decd149ef754 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a0b861fb7d042c89c1be0101dbbc735'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:25.614462 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-13e7860f-dbb2-4a7a-9b04-decd149ef754 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a0b861fb7d042c89c1be0101dbbc735'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:25.617218 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1816e1db-b603-4d0e-abfb-1f040ca08205 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:25.618877 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1816e1db-b603-4d0e-abfb-1f040ca08205 service nova] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:25.651473 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 685/1360] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:29:25 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 580 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:29:25.653019 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1816e1db-b603-4d0e-abfb-1f040ca08205 service nova] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:25.654144 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-13e7860f-dbb2-4a7a-9b04-decd149ef754 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a0b861fb7d042c89c1be0101dbbc735'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:25.657827 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-1816e1db-b603-4d0e-abfb-1f040ca08205 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:25.679552 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1816e1db-b603-4d0e-abfb-1f040ca08205 service nova] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:25.691510 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2e38ce-9475-43c5-a2a7-c51a20c6610c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:25.692173 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2e38ce-9475-43c5-a2a7-c51a20c6610c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:25.692455 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2e38ce-9475-43c5-a2a7-c51a20c6610c admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:25.692727 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2e38ce-9475-43c5-a2a7-c51a20c6610c admin admin] PATH_INFO: `/v3/users/2acb239fd3e1465faa60aa9e5701a505` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:25.693961 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2e38ce-9475-43c5-a2a7-c51a20c6610c admin admin] RBAC: Authorizing `identity:delete_user(user_id=2acb239fd3e1465faa60aa9e5701a505)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:25.695276 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-13e7860f-dbb2-4a7a-9b04-decd149ef754 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a0b861fb7d042c89c1be0101dbbc735'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:25.696066 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2e38ce-9475-43c5-a2a7-c51a20c6610c admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:25.709009 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 676/1361] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Nov 2 07:29:25 2022] DELETE /identity/v3/users/8a0b861fb7d042c89c1be0101dbbc735 => generated 0 bytes in 481 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:29:25.709009 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e7860f-dbb2-4a7a-9b04-decd149ef754 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:25.711881 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-13e7860f-dbb2-4a7a-9b04-decd149ef754 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:25.804825 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-3e2e38ce-9475-43c5-a2a7-c51a20c6610c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2acb239fd3e1465faa60aa9e5701a505'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:25.832694 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-3e2e38ce-9475-43c5-a2a7-c51a20c6610c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2acb239fd3e1465faa60aa9e5701a505'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:25.835237 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-3e2e38ce-9475-43c5-a2a7-c51a20c6610c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2acb239fd3e1465faa60aa9e5701a505'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:25.870617 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-3e2e38ce-9475-43c5-a2a7-c51a20c6610c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2acb239fd3e1465faa60aa9e5701a505'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:25.877524 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-3e2e38ce-9475-43c5-a2a7-c51a20c6610c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2acb239fd3e1465faa60aa9e5701a505'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:25.944269 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 686/1362] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Nov 2 07:29:25 2022] DELETE /identity/v3/users/2acb239fd3e1465faa60aa9e5701a505 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:29:25.945341 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5863de-0593-4fb6-a2b1-f1868910a6f0 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:25.945627 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5863de-0593-4fb6-a2b1-f1868910a6f0 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:25.945902 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5863de-0593-4fb6-a2b1-f1868910a6f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:25.998210 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-4b5863de-0593-4fb6-a2b1-f1868910a6f0 None None] MFA Rules not processed for user `e204cd785f3045ae815cd625e449a7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:29:25.999703 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-4b5863de-0593-4fb6-a2b1-f1868910a6f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:26.040864 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e7860f-dbb2-4a7a-9b04-decd149ef754 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:26.187530 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 687/1363] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:29:25 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:29:26.188886 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607fd921-2486-4d1a-a367-3fe1c0c27464 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:26.189176 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607fd921-2486-4d1a-a367-3fe1c0c27464 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:26.189894 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607fd921-2486-4d1a-a367-3fe1c0c27464 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:26.212311 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-607fd921-2486-4d1a-a367-3fe1c0c27464 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:29:26.213226 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e74c20d-6e72-43f4-b9b6-8dd11b9e4f6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:26.213706 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-607fd921-2486-4d1a-a367-3fe1c0c27464 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:26.215637 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e74c20d-6e72-43f4-b9b6-8dd11b9e4f6f admin admin] REQUEST_METHOD: `PUT` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:26.216083 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e74c20d-6e72-43f4-b9b6-8dd11b9e4f6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:26.216458 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e74c20d-6e72-43f4-b9b6-8dd11b9e4f6f admin admin] PATH_INFO: `/v3/projects/1743209f240a41f989db69285faba483/users/cf819892970e44cbad87ddf5f54aebfc/roles/b494fedd00f24f0b84a529e8672a156a` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:26.232113 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 688/1364] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:29:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:29:26.232113 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607fd921-2486-4d1a-a367-3fe1c0c27464 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:26.232113 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-607fd921-2486-4d1a-a367-3fe1c0c27464 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:26.238801 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e74c20d-6e72-43f4-b9b6-8dd11b9e4f6f admin admin] RBAC: Authorizing `identity:create_grant(project_id=1743209f240a41f989db69285faba483, user_id=cf819892970e44cbad87ddf5f54aebfc, role_id=b494fedd00f24f0b84a529e8672a156a)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:26.241943 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e74c20d-6e72-43f4-b9b6-8dd11b9e4f6f admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:26.256780 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:29:26.289769 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 677/1365] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Nov 2 07:29:25 2022] PUT /identity/v3/projects/1743209f240a41f989db69285faba483/users/cf819892970e44cbad87ddf5f54aebfc/roles/b494fedd00f24f0b84a529e8672a156a => generated 0 bytes in 576 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:29:26.294707 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06915062-7580-4f7c-9acb-d25ab668e4db None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:26.295030 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06915062-7580-4f7c-9acb-d25ab668e4db None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:26.299323 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06915062-7580-4f7c-9acb-d25ab668e4db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:26.313355 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607fd921-2486-4d1a-a367-3fe1c0c27464 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:26.339728 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-06915062-7580-4f7c-9acb-d25ab668e4db None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:29:26.345176 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-06915062-7580-4f7c-9acb-d25ab668e4db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:26.381148 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:DELETE=8,keystone:SELECT=90,keystone:INSERT=2 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:29:26.414113 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a2f3f9-12c3-4d2f-b6ce-9c03bb37de37 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43bb326fb82e4d2c83fb2eaf91860dc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:26.414679 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a2f3f9-12c3-4d2f-b6ce-9c03bb37de37 service cinder] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:26.414963 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a2f3f9-12c3-4d2f-b6ce-9c03bb37de37 service cinder] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:26.415436 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a2f3f9-12c3-4d2f-b6ce-9c03bb37de37 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:26.489819 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 678/1366] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:29:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:29:26.491157 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06915062-7580-4f7c-9acb-d25ab668e4db None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:26.497099 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-06915062-7580-4f7c-9acb-d25ab668e4db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:26.508904 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a2f3f9-12c3-4d2f-b6ce-9c03bb37de37 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:26.508904 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a2f3f9-12c3-4d2f-b6ce-9c03bb37de37 service cinder] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:26.521096 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 689/1367] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:29:26 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:29:26.521972 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a2f3f9-12c3-4d2f-b6ce-9c03bb37de37 service cinder] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:26.523378 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-87a2f3f9-12c3-4d2f-b6ce-9c03bb37de37 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:26.594882 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a2f3f9-12c3-4d2f-b6ce-9c03bb37de37 service cinder] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:26.605752 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4088f07-6692-4de3-98fb-4188ebf473c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3ec6ecb85f94ef59762877d9ea33ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:26.606445 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4088f07-6692-4de3-98fb-4188ebf473c9 service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:26.606719 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4088f07-6692-4de3-98fb-4188ebf473c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:26.606992 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4088f07-6692-4de3-98fb-4188ebf473c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:26.629031 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4088f07-6692-4de3-98fb-4188ebf473c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:26.631453 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4088f07-6692-4de3-98fb-4188ebf473c9 service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:26.643437 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 690/1368] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:29:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:29:26.644746 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6936e606-cd72-4a2e-bc31-8b2005171436 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:26.645102 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6936e606-cd72-4a2e-bc31-8b2005171436 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:26.645397 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6936e606-cd72-4a2e-bc31-8b2005171436 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:26.664442 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06915062-7580-4f7c-9acb-d25ab668e4db None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:26.673119 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e951da0-9b2b-4d11-be4c-dd86cb5d7a32 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3ec6ecb85f94ef59762877d9ea33ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:26.674507 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e951da0-9b2b-4d11-be4c-dd86cb5d7a32 service neutron] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:26.674799 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e951da0-9b2b-4d11-be4c-dd86cb5d7a32 service neutron] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:26.675077 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e951da0-9b2b-4d11-be4c-dd86cb5d7a32 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:26.689210 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-6936e606-cd72-4a2e-bc31-8b2005171436 None None] MFA Rules not processed for user `2455ebfc92ba4d398a6279d6102a0408`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:29:26.690706 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-6936e606-cd72-4a2e-bc31-8b2005171436 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:26.702728 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e951da0-9b2b-4d11-be4c-dd86cb5d7a32 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:26.705638 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e951da0-9b2b-4d11-be4c-dd86cb5d7a32 service neutron] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:26.712151 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:29:26.744894 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 679/1369] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:29:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:29:26.746750 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e951da0-9b2b-4d11-be4c-dd86cb5d7a32 service neutron] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:26.749568 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-8e951da0-9b2b-4d11-be4c-dd86cb5d7a32 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:26.772488 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e951da0-9b2b-4d11-be4c-dd86cb5d7a32 service neutron] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:26.785097 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2220c5c-0aa3-4c42-9b31-3f719a2a0bf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:26.785998 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2220c5c-0aa3-4c42-9b31-3f719a2a0bf1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:26.786297 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2220c5c-0aa3-4c42-9b31-3f719a2a0bf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:26.786924 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2220c5c-0aa3-4c42-9b31-3f719a2a0bf1 admin admin] PATH_INFO: `/v3/users/fee48a9736c94a76930f99d4b7b11646` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:26.789107 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2220c5c-0aa3-4c42-9b31-3f719a2a0bf1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fee48a9736c94a76930f99d4b7b11646)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:26.791807 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2220c5c-0aa3-4c42-9b31-3f719a2a0bf1 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:26.798528 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 691/1370] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:29:26 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:29:26.812391 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6936e606-cd72-4a2e-bc31-8b2005171436 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:26.814972 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-6936e606-cd72-4a2e-bc31-8b2005171436 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:26.902775 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-b2220c5c-0aa3-4c42-9b31-3f719a2a0bf1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fee48a9736c94a76930f99d4b7b11646'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:26.923620 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-b2220c5c-0aa3-4c42-9b31-3f719a2a0bf1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fee48a9736c94a76930f99d4b7b11646'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:26.924366 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-b2220c5c-0aa3-4c42-9b31-3f719a2a0bf1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fee48a9736c94a76930f99d4b7b11646'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:26.940565 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-b2220c5c-0aa3-4c42-9b31-3f719a2a0bf1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fee48a9736c94a76930f99d4b7b11646'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:26.975361 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-b2220c5c-0aa3-4c42-9b31-3f719a2a0bf1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fee48a9736c94a76930f99d4b7b11646'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:26.998958 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 680/1371] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Nov 2 07:29:26 2022] DELETE /identity/v3/users/fee48a9736c94a76930f99d4b7b11646 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:29:27.004389 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb167bf3-6738-4b4d-8c2c-aeb079b675c2 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:27.005368 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb167bf3-6738-4b4d-8c2c-aeb079b675c2 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:27.005650 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb167bf3-6738-4b4d-8c2c-aeb079b675c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:27.051293 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6936e606-cd72-4a2e-bc31-8b2005171436 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:27.082142 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-fb167bf3-6738-4b4d-8c2c-aeb079b675c2 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:29:27.084750 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-fb167bf3-6738-4b4d-8c2c-aeb079b675c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:27.186342 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:29:27.250693 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0190acc-6007-4521-82e3-a3b5c07be4fa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e48cba3f804cf08b2477f01ae95179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:27.251334 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0190acc-6007-4521-82e3-a3b5c07be4fa service nova] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:27.251611 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0190acc-6007-4521-82e3-a3b5c07be4fa service nova] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:27.253446 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0190acc-6007-4521-82e3-a3b5c07be4fa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:27.255539 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-e0190acc-6007-4521-82e3-a3b5c07be4fa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:27.264801 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 681/1372] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:29:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:29:27.264801 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c969febc-0da8-42df-bfb5-16df7c400dd3 None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:27.264801 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c969febc-0da8-42df-bfb5-16df7c400dd3 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:27.264801 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c969febc-0da8-42df-bfb5-16df7c400dd3 None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:27.269979 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 682/1373] 10.222.0.18 () {54 vars in 912 bytes} [Wed Nov 2 07:29:27 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:29:27.271510 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a6d90d-42ee-4493-8293-0b0c234a886f None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:27.271717 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a6d90d-42ee-4493-8293-0b0c234a886f None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:27.277800 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a6d90d-42ee-4493-8293-0b0c234a886f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:27.360998 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-c5a6d90d-42ee-4493-8293-0b0c234a886f None None] MFA Rules not processed for user `b3ec6ecb85f94ef59762877d9ea33ed0`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:29:27.363990 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-c5a6d90d-42ee-4493-8293-0b0c234a886f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:27.368113 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:29:27.395665 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0190acc-6007-4521-82e3-a3b5c07be4fa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:27.397685 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0190acc-6007-4521-82e3-a3b5c07be4fa service nova] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:27.441403 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 692/1374] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:29:26 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 630 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:29:27.444661 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0190acc-6007-4521-82e3-a3b5c07be4fa service nova] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:27.446937 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-e0190acc-6007-4521-82e3-a3b5c07be4fa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:27.505333 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 683/1375] 10.222.0.18 () {58 vars in 1009 bytes} [Wed Nov 2 07:29:27 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:29:27.514043 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a6d90d-42ee-4493-8293-0b0c234a886f None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:27.520629 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-c5a6d90d-42ee-4493-8293-0b0c234a886f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:27.549535 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a6d90d-42ee-4493-8293-0b0c234a886f None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:27.567928 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4d84aa-b899-434a-9760-d68a12a71f22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:27.569278 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4d84aa-b899-434a-9760-d68a12a71f22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:27.569726 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4d84aa-b899-434a-9760-d68a12a71f22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:27.570003 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4d84aa-b899-434a-9760-d68a12a71f22 admin admin] PATH_INFO: `/v3/projects/ba2494e16a1e474180b680c108c16af2` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:27.573087 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4d84aa-b899-434a-9760-d68a12a71f22 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ba2494e16a1e474180b680c108c16af2)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:27.575658 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4d84aa-b899-434a-9760-d68a12a71f22 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:27.647503 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0190acc-6007-4521-82e3-a3b5c07be4fa service nova] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:27.662828 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83049984-54e7-4a8f-ac09-aad88425b583 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3ec6ecb85f94ef59762877d9ea33ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:27.664215 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83049984-54e7-4a8f-ac09-aad88425b583 service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:27.664215 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83049984-54e7-4a8f-ac09-aad88425b583 service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:27.664215 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83049984-54e7-4a8f-ac09-aad88425b583 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:27.681008 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83049984-54e7-4a8f-ac09-aad88425b583 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:27.682812 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83049984-54e7-4a8f-ac09-aad88425b583 service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:27.696610 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-5e4d84aa-b899-434a-9760-d68a12a71f22 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ba2494e16a1e474180b680c108c16af2', 'request_id': 'req-5e4d84aa-b899-434a-9760-d68a12a71f22'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:27.711512 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-5e4d84aa-b899-434a-9760-d68a12a71f22 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ba2494e16a1e474180b680c108c16af2', 'request_id': 'req-5e4d84aa-b899-434a-9760-d68a12a71f22'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:27.716601 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 693/1376] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:29:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 274 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:29:27.719188 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83049984-54e7-4a8f-ac09-aad88425b583 service neutron] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:27.726183 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-5e4d84aa-b899-434a-9760-d68a12a71f22 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ba2494e16a1e474180b680c108c16af2', 'request_id': 'req-5e4d84aa-b899-434a-9760-d68a12a71f22'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:27.745461 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83049984-54e7-4a8f-ac09-aad88425b583 service neutron] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:27.754299 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-5e4d84aa-b899-434a-9760-d68a12a71f22 admin admin] The token cache is being invalidate because project ba2494e16a1e474180b680c108c16af2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81501) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:29:27.754641 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-5e4d84aa-b899-434a-9760-d68a12a71f22 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:27.755732 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-5e4d84aa-b899-434a-9760-d68a12a71f22 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:27.762440 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 684/1377] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Nov 2 07:29:27 2022] DELETE /identity/v3/projects/ba2494e16a1e474180b680c108c16af2 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:29:27.768035 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c66bd0a-f914-410b-8dca-63ab12101714 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:27.768383 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c66bd0a-f914-410b-8dca-63ab12101714 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:27.768744 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c66bd0a-f914-410b-8dca-63ab12101714 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:27.803557 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-8c66bd0a-f914-410b-8dca-63ab12101714 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:29:27.805411 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-8c66bd0a-f914-410b-8dca-63ab12101714 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:27.844339 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e1da63-0457-4422-9801-ea678152594a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3ec6ecb85f94ef59762877d9ea33ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:27.845300 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e1da63-0457-4422-9801-ea678152594a service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:27.845804 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e1da63-0457-4422-9801-ea678152594a service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:27.846193 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e1da63-0457-4422-9801-ea678152594a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:27.848793 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-a7e1da63-0457-4422-9801-ea678152594a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:27.861317 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e1da63-0457-4422-9801-ea678152594a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:27.863627 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e1da63-0457-4422-9801-ea678152594a service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:27.877063 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 694/1378] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:29:27 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:29:27.920096 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 685/1379] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:29:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:29:27.920813 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2116a998-f8f8-45d2-8477-c325bc158d33 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:27.921252 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2116a998-f8f8-45d2-8477-c325bc158d33 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:27.921700 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2116a998-f8f8-45d2-8477-c325bc158d33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:27.948488 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-2116a998-f8f8-45d2-8477-c325bc158d33 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:29:27.949966 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-2116a998-f8f8-45d2-8477-c325bc158d33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:27.963347 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 695/1380] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:29:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:29:27.965221 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c66bd0a-f914-410b-8dca-63ab12101714 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:27.970617 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:29:27.985201 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c66bd0a-f914-410b-8dca-63ab12101714 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:27.993651 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3babab7f-afd9-4f02-8c6e-714247fe42b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:27.994654 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3babab7f-afd9-4f02-8c6e-714247fe42b4 admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:27.995079 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3babab7f-afd9-4f02-8c6e-714247fe42b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:27.995481 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3babab7f-afd9-4f02-8c6e-714247fe42b4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:27.996545 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3babab7f-afd9-4f02-8c6e-714247fe42b4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:29:27.998107 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3babab7f-afd9-4f02-8c6e-714247fe42b4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:28.000621 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3babab7f-afd9-4f02-8c6e-714247fe42b4 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:28.019573 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 686/1381] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Nov 2 07:29:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:29:28.025509 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2116a998-f8f8-45d2-8477-c325bc158d33 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:28.049812 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2116a998-f8f8-45d2-8477-c325bc158d33 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:28.057007 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef0ba7a-d387-49a6-b224-35663a55868f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:28.058188 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef0ba7a-d387-49a6-b224-35663a55868f admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:28.058644 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef0ba7a-d387-49a6-b224-35663a55868f admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:28.059161 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef0ba7a-d387-49a6-b224-35663a55868f admin admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:28.060069 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef0ba7a-d387-49a6-b224-35663a55868f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:28.064184 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef0ba7a-d387-49a6-b224-35663a55868f admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:28.122177 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 696/1382] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Nov 2 07:29:28 2022] POST /identity/v3/projects => generated 383 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:29:28.123814 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3babab7f-afd9-4f02-8c6e-714247fe42b4 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:28.136508 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3babab7f-afd9-4f02-8c6e-714247fe42b4 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:28.244092 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef0ba7a-d387-49a6-b224-35663a55868f admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:28.246083 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-8ef0ba7a-d387-49a6-b224-35663a55868f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:28.258147 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ed4863-38e5-4b3e-81c1-2c2f301090cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:28.258815 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ed4863-38e5-4b3e-81c1-2c2f301090cb admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:28.259101 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ed4863-38e5-4b3e-81c1-2c2f301090cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:28.259376 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ed4863-38e5-4b3e-81c1-2c2f301090cb admin admin] PATH_INFO: `/v3/users` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:28.260018 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ed4863-38e5-4b3e-81c1-2c2f301090cb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:28.264012 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ed4863-38e5-4b3e-81c1-2c2f301090cb admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:28.275639 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef0ba7a-d387-49a6-b224-35663a55868f admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:28.291830 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f93c87-99b1-4f6f-9ef7-7568869dc698 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:28.292543 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f93c87-99b1-4f6f-9ef7-7568869dc698 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:28.292823 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f93c87-99b1-4f6f-9ef7-7568869dc698 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:28.293093 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f93c87-99b1-4f6f-9ef7-7568869dc698 admin admin] PATH_INFO: `/v3/users/1bdbe843b0bd4718955df87196c1ec66` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:28.300582 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f93c87-99b1-4f6f-9ef7-7568869dc698 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1bdbe843b0bd4718955df87196c1ec66)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:28.303110 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f93c87-99b1-4f6f-9ef7-7568869dc698 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:28.335723 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 687/1383] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Nov 2 07:29:28 2022] POST /identity/v3/users => generated 292 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 07:29:28.341046 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ed4863-38e5-4b3e-81c1-2c2f301090cb admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:28.365888 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ed4863-38e5-4b3e-81c1-2c2f301090cb admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:28.385124 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8aa12ca-180e-431d-a65e-0cad18d531d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:28.385965 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8aa12ca-180e-431d-a65e-0cad18d531d5 admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:28.386828 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8aa12ca-180e-431d-a65e-0cad18d531d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:28.387261 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8aa12ca-180e-431d-a65e-0cad18d531d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:28.388203 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8aa12ca-180e-431d-a65e-0cad18d531d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:28.390181 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8aa12ca-180e-431d-a65e-0cad18d531d5 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:28.446103 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-d7f93c87-99b1-4f6f-9ef7-7568869dc698 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1bdbe843b0bd4718955df87196c1ec66'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:28.465078 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 688/1384] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:29:28 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:29:28.468604 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8aa12ca-180e-431d-a65e-0cad18d531d5 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:28.472360 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-e8aa12ca-180e-431d-a65e-0cad18d531d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:28.484188 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-d7f93c87-99b1-4f6f-9ef7-7568869dc698 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1bdbe843b0bd4718955df87196c1ec66'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:28.484920 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-d7f93c87-99b1-4f6f-9ef7-7568869dc698 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1bdbe843b0bd4718955df87196c1ec66'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:28.487705 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:29:28.509239 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-d7f93c87-99b1-4f6f-9ef7-7568869dc698 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1bdbe843b0bd4718955df87196c1ec66'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:28.509980 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8aa12ca-180e-431d-a65e-0cad18d531d5 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:28.512152 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-d7f93c87-99b1-4f6f-9ef7-7568869dc698 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1bdbe843b0bd4718955df87196c1ec66'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:28.544034 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73075507-a742-4989-973c-62211384e8a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:28.546959 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73075507-a742-4989-973c-62211384e8a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:28.547257 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73075507-a742-4989-973c-62211384e8a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:28.559809 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 697/1385] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Nov 2 07:29:28 2022] DELETE /identity/v3/users/1bdbe843b0bd4718955df87196c1ec66 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:29:28.562664 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f93c87-99b1-4f6f-9ef7-7568869dc698 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:28.563515 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73075507-a742-4989-973c-62211384e8a6 admin admin] PATH_INFO: `/v3/projects/2bd29447b6134170804715ea2125fa7a` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:28.564418 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-d7f93c87-99b1-4f6f-9ef7-7568869dc698 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:28.567138 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73075507-a742-4989-973c-62211384e8a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2bd29447b6134170804715ea2125fa7a)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:28.569517 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73075507-a742-4989-973c-62211384e8a6 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:28.676596 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-73075507-a742-4989-973c-62211384e8a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2bd29447b6134170804715ea2125fa7a', 'request_id': 'req-73075507-a742-4989-973c-62211384e8a6'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:28.685360 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-73075507-a742-4989-973c-62211384e8a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2bd29447b6134170804715ea2125fa7a', 'request_id': 'req-73075507-a742-4989-973c-62211384e8a6'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:28.697236 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-73075507-a742-4989-973c-62211384e8a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2bd29447b6134170804715ea2125fa7a', 'request_id': 'req-73075507-a742-4989-973c-62211384e8a6'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:28.715019 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-73075507-a742-4989-973c-62211384e8a6 admin admin] The token cache is being invalidate because project 2bd29447b6134170804715ea2125fa7a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81501) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:29:28.715361 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-73075507-a742-4989-973c-62211384e8a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:28.716071 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-73075507-a742-4989-973c-62211384e8a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:28.719964 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 689/1386] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Nov 2 07:29:28 2022] DELETE /identity/v3/projects/2bd29447b6134170804715ea2125fa7a => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:29:28.721525 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84f9206-a931-44a0-8118-7d5cc2a90e51 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:28.722251 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84f9206-a931-44a0-8118-7d5cc2a90e51 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:28.722886 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84f9206-a931-44a0-8118-7d5cc2a90e51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:28.749880 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-a84f9206-a931-44a0-8118-7d5cc2a90e51 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:29:28.751686 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-a84f9206-a931-44a0-8118-7d5cc2a90e51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:28.759957 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f93c87-99b1-4f6f-9ef7-7568869dc698 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:28.853581 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a95eb74-0ad8-4fb1-8804-5182b1c971dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:28.855942 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a95eb74-0ad8-4fb1-8804-5182b1c971dc admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:28.856245 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a95eb74-0ad8-4fb1-8804-5182b1c971dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:28.856525 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a95eb74-0ad8-4fb1-8804-5182b1c971dc admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:28.857195 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a95eb74-0ad8-4fb1-8804-5182b1c971dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:28.859092 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a95eb74-0ad8-4fb1-8804-5182b1c971dc admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:28.871556 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 698/1387] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:29:28 2022] GET /identity/v3/roles => generated 1420 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:29:28.873028 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a21749-9ecb-4452-a73d-7206269c8f63 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:28.873317 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a21749-9ecb-4452-a73d-7206269c8f63 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:28.873593 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a21749-9ecb-4452-a73d-7206269c8f63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:28.903361 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:29:28.957913 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 690/1388] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:29:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:29:28.958975 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84f9206-a931-44a0-8118-7d5cc2a90e51 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:28.965797 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-49a21749-9ecb-4452-a73d-7206269c8f63 None None] MFA Rules not processed for user `17a6f8d01e6d433ca467238f07ec15ba`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:29:28.967780 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-a84f9206-a931-44a0-8118-7d5cc2a90e51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:28.971568 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-49a21749-9ecb-4452-a73d-7206269c8f63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:29.002618 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84f9206-a931-44a0-8118-7d5cc2a90e51 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:29.029208 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb672fab-b791-4044-8310-cf2aa5b764a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:29.029883 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb672fab-b791-4044-8310-cf2aa5b764a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:29.030166 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb672fab-b791-4044-8310-cf2aa5b764a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:29.030608 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb672fab-b791-4044-8310-cf2aa5b764a1 admin admin] PATH_INFO: `/v3/projects/f6181ebe47b443b9b5e57bffd9e08244` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:29.031868 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb672fab-b791-4044-8310-cf2aa5b764a1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f6181ebe47b443b9b5e57bffd9e08244)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:29.033829 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb672fab-b791-4044-8310-cf2aa5b764a1 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:29.153591 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-fb672fab-b791-4044-8310-cf2aa5b764a1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f6181ebe47b443b9b5e57bffd9e08244', 'request_id': 'req-fb672fab-b791-4044-8310-cf2aa5b764a1'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:29.161499 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-fb672fab-b791-4044-8310-cf2aa5b764a1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f6181ebe47b443b9b5e57bffd9e08244', 'request_id': 'req-fb672fab-b791-4044-8310-cf2aa5b764a1'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:29.165272 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 699/1389] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:29:28 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 293 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:29:29.173683 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-fb672fab-b791-4044-8310-cf2aa5b764a1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f6181ebe47b443b9b5e57bffd9e08244', 'request_id': 'req-fb672fab-b791-4044-8310-cf2aa5b764a1'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:29.176659 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a21749-9ecb-4452-a73d-7206269c8f63 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:29.186885 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-49a21749-9ecb-4452-a73d-7206269c8f63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:29.189906 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-fb672fab-b791-4044-8310-cf2aa5b764a1 admin admin] The token cache is being invalidate because project f6181ebe47b443b9b5e57bffd9e08244 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81501) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:29:29.190213 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-fb672fab-b791-4044-8310-cf2aa5b764a1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:29.190944 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-fb672fab-b791-4044-8310-cf2aa5b764a1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:29.196153 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 691/1390] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Nov 2 07:29:28 2022] DELETE /identity/v3/projects/f6181ebe47b443b9b5e57bffd9e08244 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:29:29.196990 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb672fab-b791-4044-8310-cf2aa5b764a1 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:29.198713 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-fb672fab-b791-4044-8310-cf2aa5b764a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:29.202797 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a21749-9ecb-4452-a73d-7206269c8f63 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:29.207553 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:29:29.317103 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb672fab-b791-4044-8310-cf2aa5b764a1 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:29.327099 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c43a01a-a486-446f-91c4-ab2dafd84c5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:29.328077 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c43a01a-a486-446f-91c4-ab2dafd84c5f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:29.328077 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c43a01a-a486-446f-91c4-ab2dafd84c5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:29.328358 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c43a01a-a486-446f-91c4-ab2dafd84c5f admin admin] PATH_INFO: `/v3/users/e204cd785f3045ae815cd625e449a7e8` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:29.329984 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c43a01a-a486-446f-91c4-ab2dafd84c5f admin admin] RBAC: Authorizing `identity:delete_user(user_id=e204cd785f3045ae815cd625e449a7e8)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:29.332676 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c43a01a-a486-446f-91c4-ab2dafd84c5f admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:29.333485 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac21952-8bd6-47aa-958f-5c45be4e2480 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:29.334950 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac21952-8bd6-47aa-958f-5c45be4e2480 admin admin] REQUEST_METHOD: `PUT` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:29.336916 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac21952-8bd6-47aa-958f-5c45be4e2480 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:29.337947 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac21952-8bd6-47aa-958f-5c45be4e2480 admin admin] PATH_INFO: `/v3/projects/afc08e9049be4671be109b98305098a7/users/c774a9a98614453eadd676a7b51c72c7/roles/b494fedd00f24f0b84a529e8672a156a` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:29.381238 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac21952-8bd6-47aa-958f-5c45be4e2480 admin admin] RBAC: Authorizing `identity:create_grant(project_id=afc08e9049be4671be109b98305098a7, user_id=c774a9a98614453eadd676a7b51c72c7, role_id=b494fedd00f24f0b84a529e8672a156a)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:29.384157 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac21952-8bd6-47aa-958f-5c45be4e2480 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:29.419927 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 692/1391] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Nov 2 07:29:29 2022] PUT /identity/v3/projects/afc08e9049be4671be109b98305098a7/users/c774a9a98614453eadd676a7b51c72c7/roles/b494fedd00f24f0b84a529e8672a156a => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:29:29.423737 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac21952-8bd6-47aa-958f-5c45be4e2480 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:29.428256 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-eac21952-8bd6-47aa-958f-5c45be4e2480 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:29.431287 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-2c43a01a-a486-446f-91c4-ab2dafd84c5f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e204cd785f3045ae815cd625e449a7e8'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:29.454544 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-2c43a01a-a486-446f-91c4-ab2dafd84c5f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e204cd785f3045ae815cd625e449a7e8'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:29.455828 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-2c43a01a-a486-446f-91c4-ab2dafd84c5f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e204cd785f3045ae815cd625e449a7e8'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:29.466660 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-2c43a01a-a486-446f-91c4-ab2dafd84c5f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e204cd785f3045ae815cd625e449a7e8'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:29.467419 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-2c43a01a-a486-446f-91c4-ab2dafd84c5f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e204cd785f3045ae815cd625e449a7e8'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:29.528120 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 700/1392] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Nov 2 07:29:29 2022] DELETE /identity/v3/users/e204cd785f3045ae815cd625e449a7e8 => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:29:29.528120 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c43a01a-a486-446f-91c4-ab2dafd84c5f admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:29.533834 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-2c43a01a-a486-446f-91c4-ab2dafd84c5f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:29.576947 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac21952-8bd6-47aa-958f-5c45be4e2480 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:29.600185 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:29:29.742236 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fcd25f-7307-4e21-a43a-154cf37b5229 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3ec6ecb85f94ef59762877d9ea33ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:29.743950 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fcd25f-7307-4e21-a43a-154cf37b5229 service neutron] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:29.744238 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fcd25f-7307-4e21-a43a-154cf37b5229 service neutron] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:29.744513 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fcd25f-7307-4e21-a43a-154cf37b5229 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:29.749710 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-22fcd25f-7307-4e21-a43a-154cf37b5229 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:29.765326 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c43a01a-a486-446f-91c4-ab2dafd84c5f admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:29.780324 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bba134e-2ce5-493d-98c4-b1d7aec570e3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2fc6a156c54d0c9d325901d0f7f6fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:29.782070 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bba134e-2ce5-493d-98c4-b1d7aec570e3 service placement] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:29.782070 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bba134e-2ce5-493d-98c4-b1d7aec570e3 service placement] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:29.782070 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bba134e-2ce5-493d-98c4-b1d7aec570e3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:29.785181 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-1bba134e-2ce5-493d-98c4-b1d7aec570e3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:29.816855 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bba134e-2ce5-493d-98c4-b1d7aec570e3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:29.819587 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bba134e-2ce5-493d-98c4-b1d7aec570e3 service placement] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:29.843221 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 701/1393] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:29:29 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:29:29.861921 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696e0df2-df51-4b9e-acf2-529696ab44d1 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:29.862209 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696e0df2-df51-4b9e-acf2-529696ab44d1 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:29.864650 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696e0df2-df51-4b9e-acf2-529696ab44d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:29.932555 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fcd25f-7307-4e21-a43a-154cf37b5229 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:29.935205 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fcd25f-7307-4e21-a43a-154cf37b5229 service neutron] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:29.945512 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-696e0df2-df51-4b9e-acf2-529696ab44d1 None None] MFA Rules not processed for user `17a6f8d01e6d433ca467238f07ec15ba`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:29:29.952165 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-696e0df2-df51-4b9e-acf2-529696ab44d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:29.953284 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 693/1394] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:29:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 533 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:29:29.957694 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280fadc0-15eb-495f-b926-89f2ae520c19 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:29.959283 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280fadc0-15eb-495f-b926-89f2ae520c19 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:29.959736 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280fadc0-15eb-495f-b926-89f2ae520c19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:29.989030 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:29:30.003563 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-280fadc0-15eb-495f-b926-89f2ae520c19 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:29:30.006068 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-280fadc0-15eb-495f-b926-89f2ae520c19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:30.040589 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 694/1395] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:29:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:29:30.042161 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7282191a-ff65-46d4-a272-eddb8eb464cc None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:30.044617 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7282191a-ff65-46d4-a272-eddb8eb464cc None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:30.046907 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7282191a-ff65-46d4-a272-eddb8eb464cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:30.082055 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-7282191a-ff65-46d4-a272-eddb8eb464cc None None] MFA Rules not processed for user `0e571b0f4d6f4fafb71939f501b8deec`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:29:30.083447 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-7282191a-ff65-46d4-a272-eddb8eb464cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:30.150015 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 702/1396] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:29:29 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 306 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:29:30.154063 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac2ef7f-8c70-4f7c-83b7-0d4ee6ad1c58 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:30.155106 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac2ef7f-8c70-4f7c-83b7-0d4ee6ad1c58 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:30.155106 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac2ef7f-8c70-4f7c-83b7-0d4ee6ad1c58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:30.241214 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 695/1397] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:29:30 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:29:30.241214 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2804b7-644b-4205-988f-dc1e8f21dcb4 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:30.241214 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2804b7-644b-4205-988f-dc1e8f21dcb4 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:30.241214 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2804b7-644b-4205-988f-dc1e8f21dcb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:30.241907 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-3ac2ef7f-8c70-4f7c-83b7-0d4ee6ad1c58 None None] MFA Rules not processed for user `c774a9a98614453eadd676a7b51c72c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:29:30.242904 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-3ac2ef7f-8c70-4f7c-83b7-0d4ee6ad1c58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:30.356592 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-7c2804b7-644b-4205-988f-dc1e8f21dcb4 None None] MFA Rules not processed for user `cf819892970e44cbad87ddf5f54aebfc`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:29:30.364285 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-7c2804b7-644b-4205-988f-dc1e8f21dcb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:30.504998 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:29:30.509132 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 703/1398] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:29:30 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 358 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:29:30.512536 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75493b7-4efc-4150-a534-42f78fb18e67 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:30.513003 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75493b7-4efc-4150-a534-42f78fb18e67 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:30.513287 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75493b7-4efc-4150-a534-42f78fb18e67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:30.536133 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 696/1399] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:29:30 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 297 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:29:30.537291 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2804b7-644b-4205-988f-dc1e8f21dcb4 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:30.539951 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-7c2804b7-644b-4205-988f-dc1e8f21dcb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:30.565578 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-f75493b7-4efc-4150-a534-42f78fb18e67 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:29:30.566848 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-f75493b7-4efc-4150-a534-42f78fb18e67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:30.585298 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 704/1400] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:29:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:29:30.590551 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75493b7-4efc-4150-a534-42f78fb18e67 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:30.593381 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-f75493b7-4efc-4150-a534-42f78fb18e67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:30.626052 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75493b7-4efc-4150-a534-42f78fb18e67 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:30.656587 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddee103c-da8c-468b-9e13-b46152fabf7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:30.657251 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddee103c-da8c-468b-9e13-b46152fabf7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:30.657670 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddee103c-da8c-468b-9e13-b46152fabf7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:30.657760 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddee103c-da8c-468b-9e13-b46152fabf7e admin admin] PATH_INFO: `/v3/projects/44c8892cee764351b8eaa1eb4187cdc3` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:30.658955 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddee103c-da8c-468b-9e13-b46152fabf7e admin admin] RBAC: Authorizing `identity:delete_project(project_id=44c8892cee764351b8eaa1eb4187cdc3)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:30.659967 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: WARNING py.warnings [None req-ddee103c-da8c-468b-9e13-b46152fabf7e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Nov 02 07:29:30.659967 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: warnings.warn(msg) Nov 02 07:29:30.659967 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: Nov 02 07:29:30.661162 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddee103c-da8c-468b-9e13-b46152fabf7e admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:30.701260 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2804b7-644b-4205-988f-dc1e8f21dcb4 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:30.720582 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eecad95-a9c1-4f63-81f8-fc31e0e5cfda service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e48cba3f804cf08b2477f01ae95179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:30.721085 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eecad95-a9c1-4f63-81f8-fc31e0e5cfda service nova] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:30.721465 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eecad95-a9c1-4f63-81f8-fc31e0e5cfda service nova] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:30.722248 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eecad95-a9c1-4f63-81f8-fc31e0e5cfda service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:30.727952 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-3eecad95-a9c1-4f63-81f8-fc31e0e5cfda service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:30.756872 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eecad95-a9c1-4f63-81f8-fc31e0e5cfda service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:30.758933 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eecad95-a9c1-4f63-81f8-fc31e0e5cfda service nova] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:30.797753 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 697/1401] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:29:30 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 257 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:29:30.798938 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eecad95-a9c1-4f63-81f8-fc31e0e5cfda service nova] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:30.804985 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-3eecad95-a9c1-4f63-81f8-fc31e0e5cfda service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:30.818764 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-ddee103c-da8c-468b-9e13-b46152fabf7e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '44c8892cee764351b8eaa1eb4187cdc3', 'request_id': 'req-ddee103c-da8c-468b-9e13-b46152fabf7e'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:30.846196 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-ddee103c-da8c-468b-9e13-b46152fabf7e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '44c8892cee764351b8eaa1eb4187cdc3', 'request_id': 'req-ddee103c-da8c-468b-9e13-b46152fabf7e'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:30.849423 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eecad95-a9c1-4f63-81f8-fc31e0e5cfda service nova] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:30.860063 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-ddee103c-da8c-468b-9e13-b46152fabf7e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '44c8892cee764351b8eaa1eb4187cdc3', 'request_id': 'req-ddee103c-da8c-468b-9e13-b46152fabf7e'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:30.899212 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-ddee103c-da8c-468b-9e13-b46152fabf7e admin admin] The token cache is being invalidate because project 44c8892cee764351b8eaa1eb4187cdc3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:29:30.899548 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-ddee103c-da8c-468b-9e13-b46152fabf7e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:30.900249 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-ddee103c-da8c-468b-9e13-b46152fabf7e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:30.910883 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 705/1402] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Nov 2 07:29:30 2022] DELETE /identity/v3/projects/44c8892cee764351b8eaa1eb4187cdc3 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:29:30.911776 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddee103c-da8c-468b-9e13-b46152fabf7e admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:30.915043 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-ddee103c-da8c-468b-9e13-b46152fabf7e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:30.983763 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:29:31.027040 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7a8e55-0a83-440a-8429-71b272da66fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:31.027767 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7a8e55-0a83-440a-8429-71b272da66fa admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:31.028195 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7a8e55-0a83-440a-8429-71b272da66fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:31.028479 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7a8e55-0a83-440a-8429-71b272da66fa admin admin] PATH_INFO: `/v3/projects` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:31.029123 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7a8e55-0a83-440a-8429-71b272da66fa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:31.037225 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7a8e55-0a83-440a-8429-71b272da66fa admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:31.099338 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddee103c-da8c-468b-9e13-b46152fabf7e admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:31.108547 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91d831c-46d5-483f-838c-80cf7a2d11d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:31.117725 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91d831c-46d5-483f-838c-80cf7a2d11d6 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:31.118032 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91d831c-46d5-483f-838c-80cf7a2d11d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:31.118297 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91d831c-46d5-483f-838c-80cf7a2d11d6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:31.119049 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91d831c-46d5-483f-838c-80cf7a2d11d6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:31.129605 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91d831c-46d5-483f-838c-80cf7a2d11d6 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:31.133816 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 698/1403] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Nov 2 07:29:30 2022] POST /identity/v3/projects => generated 381 bytes in 335 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:29:31.134688 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7a8e55-0a83-440a-8429-71b272da66fa admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:31.138829 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-8d7a8e55-0a83-440a-8429-71b272da66fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:31.202797 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 706/1404] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Nov 2 07:29:30 2022] POST /identity/v3/projects => generated 405 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:29:31.207175 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec4d39c-6590-4d98-aba4-0bbcc2fc1d8c None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:31.207403 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec4d39c-6590-4d98-aba4-0bbcc2fc1d8c None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:31.207680 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec4d39c-6590-4d98-aba4-0bbcc2fc1d8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:31.255544 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-fec4d39c-6590-4d98-aba4-0bbcc2fc1d8c None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:29:31.257061 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-fec4d39c-6590-4d98-aba4-0bbcc2fc1d8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:31.307430 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7a8e55-0a83-440a-8429-71b272da66fa admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:31.320504 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:29:31.376292 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 707/1405] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:29:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:29:31.377809 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec4d39c-6590-4d98-aba4-0bbcc2fc1d8c None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:31.383424 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-fec4d39c-6590-4d98-aba4-0bbcc2fc1d8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:31.404297 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec4d39c-6590-4d98-aba4-0bbcc2fc1d8c None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:31.411630 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8ade74-1229-4172-87bf-4b59fc16592a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:31.413369 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8ade74-1229-4172-87bf-4b59fc16592a admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:31.413872 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8ade74-1229-4172-87bf-4b59fc16592a admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:31.414532 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8ade74-1229-4172-87bf-4b59fc16592a admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:31.415589 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8ade74-1229-4172-87bf-4b59fc16592a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:31.419198 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8ade74-1229-4172-87bf-4b59fc16592a admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:31.460442 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313cd0b4-820f-451a-9bd7-c7b6fcc444f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:31.461027 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313cd0b4-820f-451a-9bd7-c7b6fcc444f0 admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:31.461249 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313cd0b4-820f-451a-9bd7-c7b6fcc444f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:31.461531 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313cd0b4-820f-451a-9bd7-c7b6fcc444f0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:31.462170 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313cd0b4-820f-451a-9bd7-c7b6fcc444f0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:29:31.462505 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313cd0b4-820f-451a-9bd7-c7b6fcc444f0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:31.464173 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313cd0b4-820f-451a-9bd7-c7b6fcc444f0 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:31.465985 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 708/1406] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Nov 2 07:29:31 2022] POST /identity/v3/users => generated 297 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:29:31.467225 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8ade74-1229-4172-87bf-4b59fc16592a admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:31.499316 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8ade74-1229-4172-87bf-4b59fc16592a admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:31.506090 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 699/1407] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Nov 2 07:29:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 372 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:29:31.508201 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118919de-354c-4ee9-ac92-48bcc5f2bf0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:31.508974 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b307a2-1398-4ddb-b96c-28ecf1e7da16 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:31.509263 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b307a2-1398-4ddb-b96c-28ecf1e7da16 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:31.509625 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b307a2-1398-4ddb-b96c-28ecf1e7da16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:31.510140 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118919de-354c-4ee9-ac92-48bcc5f2bf0a admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:31.513043 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118919de-354c-4ee9-ac92-48bcc5f2bf0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:31.513992 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118919de-354c-4ee9-ac92-48bcc5f2bf0a admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:31.515655 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118919de-354c-4ee9-ac92-48bcc5f2bf0a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:31.518353 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118919de-354c-4ee9-ac92-48bcc5f2bf0a admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:31.545944 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-49b307a2-1398-4ddb-b96c-28ecf1e7da16 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:29:31.547368 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-49b307a2-1398-4ddb-b96c-28ecf1e7da16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:31.564074 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 709/1408] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Nov 2 07:29:31 2022] POST /identity/v3/users => generated 309 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:29:31.565404 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118919de-354c-4ee9-ac92-48bcc5f2bf0a admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:31.579853 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 700/1409] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:29:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:29:31.584124 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118919de-354c-4ee9-ac92-48bcc5f2bf0a admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:31.585362 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b307a2-1398-4ddb-b96c-28ecf1e7da16 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:31.591464 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6b27f5-4d45-4148-a7f2-76caf519f67f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:31.592312 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6b27f5-4d45-4148-a7f2-76caf519f67f admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:31.592751 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6b27f5-4d45-4148-a7f2-76caf519f67f admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:31.593115 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6b27f5-4d45-4148-a7f2-76caf519f67f admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:31.594294 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6b27f5-4d45-4148-a7f2-76caf519f67f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:31.596229 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6b27f5-4d45-4148-a7f2-76caf519f67f admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:31.615848 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 710/1410] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:29:31 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:29:31.616029 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b307a2-1398-4ddb-b96c-28ecf1e7da16 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:31.616135 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6b27f5-4d45-4148-a7f2-76caf519f67f admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:31.639250 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96bc8de-b370-4192-82bd-38a2db819d43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:31.640098 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96bc8de-b370-4192-82bd-38a2db819d43 admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:31.654203 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96bc8de-b370-4192-82bd-38a2db819d43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:31.654660 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96bc8de-b370-4192-82bd-38a2db819d43 admin admin] PATH_INFO: `/v3/projects` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:31.656470 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96bc8de-b370-4192-82bd-38a2db819d43 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:31.659148 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6b27f5-4d45-4148-a7f2-76caf519f67f admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:31.661626 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96bc8de-b370-4192-82bd-38a2db819d43 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:31.673482 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e840b1e6-72b2-4e82-a2a0-123b93245203 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:31.674227 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e840b1e6-72b2-4e82-a2a0-123b93245203 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:31.675743 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e840b1e6-72b2-4e82-a2a0-123b93245203 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:31.676476 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e840b1e6-72b2-4e82-a2a0-123b93245203 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:31.677466 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e840b1e6-72b2-4e82-a2a0-123b93245203 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:31.680326 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e840b1e6-72b2-4e82-a2a0-123b93245203 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:31.711039 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 711/1411] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:29:31 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:29:31.713145 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e840b1e6-72b2-4e82-a2a0-123b93245203 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:31.745294 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 701/1412] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Nov 2 07:29:31 2022] POST /identity/v3/projects => generated 371 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:29:31.745294 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96bc8de-b370-4192-82bd-38a2db819d43 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:31.747627 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e840b1e6-72b2-4e82-a2a0-123b93245203 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:31.864129 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:29:31.897895 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a15194-071f-45f4-b7cb-6c869a3b74c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:31.898587 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a15194-071f-45f4-b7cb-6c869a3b74c1 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:31.898870 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a15194-071f-45f4-b7cb-6c869a3b74c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:31.902351 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a15194-071f-45f4-b7cb-6c869a3b74c1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:31.910244 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a15194-071f-45f4-b7cb-6c869a3b74c1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:29:31.912471 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a15194-071f-45f4-b7cb-6c869a3b74c1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:31.914329 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a15194-071f-45f4-b7cb-6c869a3b74c1 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:31.929974 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96bc8de-b370-4192-82bd-38a2db819d43 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:31.949297 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4c0396-97db-40ba-8d66-f92d69241ce4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:31.950205 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4c0396-97db-40ba-8d66-f92d69241ce4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:31.950580 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4c0396-97db-40ba-8d66-f92d69241ce4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:31.950917 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4c0396-97db-40ba-8d66-f92d69241ce4 admin admin] PATH_INFO: `/v3/projects/d1eab68550f348c78e18ba704c313f82/users/ccf803f8277740dca79913ee2f8b462b/roles/c4bf019fc6c44c71a6c984f6d0bab75f` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:31.970843 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 712/1413] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Nov 2 07:29:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:29:31.971948 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b8d952-f716-4ffd-8bd2-13a9740a3963 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:31.976565 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b8d952-f716-4ffd-8bd2-13a9740a3963 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:31.976855 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b8d952-f716-4ffd-8bd2-13a9740a3963 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:31.978310 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 713/1414] 10.222.0.18 () {54 vars in 912 bytes} [Wed Nov 2 07:29:31 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:29:31.979152 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b8d952-f716-4ffd-8bd2-13a9740a3963 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:32.008544 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4c0396-97db-40ba-8d66-f92d69241ce4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d1eab68550f348c78e18ba704c313f82, user_id=ccf803f8277740dca79913ee2f8b462b, role_id=c4bf019fc6c44c71a6c984f6d0bab75f)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:32.012329 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4c0396-97db-40ba-8d66-f92d69241ce4 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:32.019176 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b8d952-f716-4ffd-8bd2-13a9740a3963 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:32.029102 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6921bdc-afb9-4ced-84b2-daaa375fcc20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:32.029978 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6921bdc-afb9-4ced-84b2-daaa375fcc20 admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:32.031040 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6921bdc-afb9-4ced-84b2-daaa375fcc20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:32.033288 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6921bdc-afb9-4ced-84b2-daaa375fcc20 admin admin] PATH_INFO: `/v3/projects/718f8ac6458a47f18440c145482ca9f8/users/709bda2cecbb40318182b0db9d7a0901/roles/c4bf019fc6c44c71a6c984f6d0bab75f` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:32.049272 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 702/1415] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Nov 2 07:29:31 2022] PUT /identity/v3/projects/d1eab68550f348c78e18ba704c313f82/users/ccf803f8277740dca79913ee2f8b462b/roles/c4bf019fc6c44c71a6c984f6d0bab75f => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:29:32.058445 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4c0396-97db-40ba-8d66-f92d69241ce4 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:32.083251 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6921bdc-afb9-4ced-84b2-daaa375fcc20 admin admin] RBAC: Authorizing `identity:create_grant(project_id=718f8ac6458a47f18440c145482ca9f8, user_id=709bda2cecbb40318182b0db9d7a0901, role_id=c4bf019fc6c44c71a6c984f6d0bab75f)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:32.086516 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6921bdc-afb9-4ced-84b2-daaa375fcc20 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:32.132639 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 714/1416] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Nov 2 07:29:31 2022] PUT /identity/v3/projects/718f8ac6458a47f18440c145482ca9f8/users/709bda2cecbb40318182b0db9d7a0901/roles/c4bf019fc6c44c71a6c984f6d0bab75f => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:29:32.132639 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6921bdc-afb9-4ced-84b2-daaa375fcc20 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:32.135288 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-e6921bdc-afb9-4ced-84b2-daaa375fcc20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:32.191493 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:29:32.197065 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4c0396-97db-40ba-8d66-f92d69241ce4 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:32.270666 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6921bdc-afb9-4ced-84b2-daaa375fcc20 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:32.282962 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef04f07-a2f3-4e58-9bd2-e0a10168abd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:32.286298 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef04f07-a2f3-4e58-9bd2-e0a10168abd7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:32.286298 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef04f07-a2f3-4e58-9bd2-e0a10168abd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:32.286298 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef04f07-a2f3-4e58-9bd2-e0a10168abd7 admin admin] PATH_INFO: `/v3/projects/818e1198e7c64603bb957c4ed7198b45` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:32.293077 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef04f07-a2f3-4e58-9bd2-e0a10168abd7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=818e1198e7c64603bb957c4ed7198b45)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:32.297288 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef04f07-a2f3-4e58-9bd2-e0a10168abd7 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:32.381871 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac06a6b-cd3f-434e-ad5b-8bbd01989857 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:32.383505 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac06a6b-cd3f-434e-ad5b-8bbd01989857 admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:32.385090 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac06a6b-cd3f-434e-ad5b-8bbd01989857 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:32.385605 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac06a6b-cd3f-434e-ad5b-8bbd01989857 admin admin] PATH_INFO: `/v3/users` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:32.387289 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac06a6b-cd3f-434e-ad5b-8bbd01989857 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:32.390495 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac06a6b-cd3f-434e-ad5b-8bbd01989857 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:32.430072 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-7ef04f07-a2f3-4e58-9bd2-e0a10168abd7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '818e1198e7c64603bb957c4ed7198b45', 'request_id': 'req-7ef04f07-a2f3-4e58-9bd2-e0a10168abd7'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:32.441379 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-7ef04f07-a2f3-4e58-9bd2-e0a10168abd7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '818e1198e7c64603bb957c4ed7198b45', 'request_id': 'req-7ef04f07-a2f3-4e58-9bd2-e0a10168abd7'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:32.449579 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-7ef04f07-a2f3-4e58-9bd2-e0a10168abd7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '818e1198e7c64603bb957c4ed7198b45', 'request_id': 'req-7ef04f07-a2f3-4e58-9bd2-e0a10168abd7'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:32.454153 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 703/1417] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Nov 2 07:29:32 2022] POST /identity/v3/users => generated 286 bytes in 404 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:29:32.455188 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac06a6b-cd3f-434e-ad5b-8bbd01989857 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:32.476699 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-7ef04f07-a2f3-4e58-9bd2-e0a10168abd7 admin admin] The token cache is being invalidate because project 818e1198e7c64603bb957c4ed7198b45 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:29:32.477172 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-7ef04f07-a2f3-4e58-9bd2-e0a10168abd7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:32.478046 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-7ef04f07-a2f3-4e58-9bd2-e0a10168abd7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:32.478883 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac06a6b-cd3f-434e-ad5b-8bbd01989857 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:32.484326 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 715/1418] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Nov 2 07:29:32 2022] DELETE /identity/v3/projects/818e1198e7c64603bb957c4ed7198b45 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:29:32.486670 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65f2d78-df4d-40a0-8df8-c6bb8ca44110 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:32.488651 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65f2d78-df4d-40a0-8df8-c6bb8ca44110 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:32.489044 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65f2d78-df4d-40a0-8df8-c6bb8ca44110 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:32.553845 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-a65f2d78-df4d-40a0-8df8-c6bb8ca44110 None None] MFA Rules not processed for user `b3ec6ecb85f94ef59762877d9ea33ed0`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:29:32.558721 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-a65f2d78-df4d-40a0-8df8-c6bb8ca44110 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:32.624230 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee008f43-dedd-4c21-b7c5-adfb2a1c1d44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:32.624874 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee008f43-dedd-4c21-b7c5-adfb2a1c1d44 admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:32.625159 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee008f43-dedd-4c21-b7c5-adfb2a1c1d44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:32.625434 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee008f43-dedd-4c21-b7c5-adfb2a1c1d44 admin admin] PATH_INFO: `/v3/projects` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:32.626080 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee008f43-dedd-4c21-b7c5-adfb2a1c1d44 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:32.628170 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee008f43-dedd-4c21-b7c5-adfb2a1c1d44 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:32.672872 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 704/1419] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Nov 2 07:29:32 2022] POST /identity/v3/projects => generated 379 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:29:32.672872 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee008f43-dedd-4c21-b7c5-adfb2a1c1d44 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:32.675464 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-ee008f43-dedd-4c21-b7c5-adfb2a1c1d44 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:32.677042 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 716/1420] 10.222.0.18 () {58 vars in 1009 bytes} [Wed Nov 2 07:29:32 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:29:32.678484 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65f2d78-df4d-40a0-8df8-c6bb8ca44110 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:32.679950 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-a65f2d78-df4d-40a0-8df8-c6bb8ca44110 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:32.691994 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:29:32.699238 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65f2d78-df4d-40a0-8df8-c6bb8ca44110 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:32.789668 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee008f43-dedd-4c21-b7c5-adfb2a1c1d44 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:32.796095 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698035eb-3bff-46ee-a52a-e980e9a06e9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:32.796915 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698035eb-3bff-46ee-a52a-e980e9a06e9c admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:32.797197 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698035eb-3bff-46ee-a52a-e980e9a06e9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:32.797483 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698035eb-3bff-46ee-a52a-e980e9a06e9c admin admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:32.798485 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698035eb-3bff-46ee-a52a-e980e9a06e9c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:32.800409 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698035eb-3bff-46ee-a52a-e980e9a06e9c admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:32.816910 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72577793-6aad-4faa-b4aa-851c80f31292 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:32.817919 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72577793-6aad-4faa-b4aa-851c80f31292 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:32.818264 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72577793-6aad-4faa-b4aa-851c80f31292 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:32.825491 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72577793-6aad-4faa-b4aa-851c80f31292 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:32.828565 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72577793-6aad-4faa-b4aa-851c80f31292 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:32.833653 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 705/1421] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:29:32 2022] GET /identity/v3/roles => generated 1420 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:29:32.833814 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72577793-6aad-4faa-b4aa-851c80f31292 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:32.834461 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698035eb-3bff-46ee-a52a-e980e9a06e9c admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:32.837580 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-698035eb-3bff-46ee-a52a-e980e9a06e9c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:32.860204 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698035eb-3bff-46ee-a52a-e980e9a06e9c admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:32.862163 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 717/1422] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:29:32 2022] GET /identity/v3/roles => generated 1420 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:29:32.863795 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258466e0-cbac-4913-b7a5-50ad969fb723 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:32.864180 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258466e0-cbac-4913-b7a5-50ad969fb723 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:32.864451 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258466e0-cbac-4913-b7a5-50ad969fb723 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:32.866137 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ddfc79-c82a-4145-a6ae-ce0f3bfe23b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:32.867068 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ddfc79-c82a-4145-a6ae-ce0f3bfe23b4 admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:32.867597 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ddfc79-c82a-4145-a6ae-ce0f3bfe23b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:32.867883 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ddfc79-c82a-4145-a6ae-ce0f3bfe23b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:32.868734 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ddfc79-c82a-4145-a6ae-ce0f3bfe23b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:32.871217 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ddfc79-c82a-4145-a6ae-ce0f3bfe23b4 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:32.884329 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 706/1423] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:29:32 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:29:32.885132 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ddfc79-c82a-4145-a6ae-ce0f3bfe23b4 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:32.885510 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-258466e0-cbac-4913-b7a5-50ad969fb723 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:29:32.887198 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-36ddfc79-c82a-4145-a6ae-ce0f3bfe23b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:32.887603 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-258466e0-cbac-4913-b7a5-50ad969fb723 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:32.897007 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 718/1424] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:29:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:29:32.898257 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258466e0-cbac-4913-b7a5-50ad969fb723 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:32.904870 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:29:32.907103 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-258466e0-cbac-4913-b7a5-50ad969fb723 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:32.910273 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ddfc79-c82a-4145-a6ae-ce0f3bfe23b4 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:32.917813 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50acc9cb-4394-48a8-aafe-8bf76a85258b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:32.918915 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50acc9cb-4394-48a8-aafe-8bf76a85258b admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:32.919647 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50acc9cb-4394-48a8-aafe-8bf76a85258b admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:32.920183 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50acc9cb-4394-48a8-aafe-8bf76a85258b admin admin] PATH_INFO: `/v3/users` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:32.921102 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50acc9cb-4394-48a8-aafe-8bf76a85258b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:32.923567 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50acc9cb-4394-48a8-aafe-8bf76a85258b admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:32.951487 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 707/1425] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Nov 2 07:29:32 2022] POST /identity/v3/users => generated 290 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:29:32.952451 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50acc9cb-4394-48a8-aafe-8bf76a85258b admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:32.963266 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50acc9cb-4394-48a8-aafe-8bf76a85258b admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:32.968520 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef616c5a-9efa-4d5f-adcb-8aa7000752ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:32.969407 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef616c5a-9efa-4d5f-adcb-8aa7000752ba admin admin] REQUEST_METHOD: `PUT` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:32.969901 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef616c5a-9efa-4d5f-adcb-8aa7000752ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:32.970308 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef616c5a-9efa-4d5f-adcb-8aa7000752ba admin admin] PATH_INFO: `/v3/projects/d1eab68550f348c78e18ba704c313f82/users/ccf803f8277740dca79913ee2f8b462b/roles/c4bf019fc6c44c71a6c984f6d0bab75f` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:32.973165 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef616c5a-9efa-4d5f-adcb-8aa7000752ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=d1eab68550f348c78e18ba704c313f82, user_id=ccf803f8277740dca79913ee2f8b462b, role_id=c4bf019fc6c44c71a6c984f6d0bab75f)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:32.976695 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef616c5a-9efa-4d5f-adcb-8aa7000752ba admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:33.010238 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 708/1426] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Nov 2 07:29:32 2022] PUT /identity/v3/projects/d1eab68550f348c78e18ba704c313f82/users/ccf803f8277740dca79913ee2f8b462b/roles/c4bf019fc6c44c71a6c984f6d0bab75f => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:29:33.012211 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef616c5a-9efa-4d5f-adcb-8aa7000752ba admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:33.025490 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258466e0-cbac-4913-b7a5-50ad969fb723 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:33.149520 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301479a8-c324-4072-a9a5-74b9465d3d2d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3ec6ecb85f94ef59762877d9ea33ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:33.150201 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301479a8-c324-4072-a9a5-74b9465d3d2d service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:33.150589 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301479a8-c324-4072-a9a5-74b9465d3d2d service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:33.150870 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301479a8-c324-4072-a9a5-74b9465d3d2d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:33.163084 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301479a8-c324-4072-a9a5-74b9465d3d2d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:33.165385 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301479a8-c324-4072-a9a5-74b9465d3d2d service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:33.180990 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 719/1427] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:29:32 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 283 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:29:33.183433 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301479a8-c324-4072-a9a5-74b9465d3d2d service neutron] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:33.193838 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef616c5a-9efa-4d5f-adcb-8aa7000752ba admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:33.204358 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74978779-8485-4fee-875a-05d5327bfba4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:33.207611 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74978779-8485-4fee-875a-05d5327bfba4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:33.207893 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74978779-8485-4fee-875a-05d5327bfba4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:33.208166 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74978779-8485-4fee-875a-05d5327bfba4 admin admin] PATH_INFO: `/v3/projects/718f8ac6458a47f18440c145482ca9f8/users/709bda2cecbb40318182b0db9d7a0901/roles/c4bf019fc6c44c71a6c984f6d0bab75f` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:33.211088 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301479a8-c324-4072-a9a5-74b9465d3d2d service neutron] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:33.213720 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74978779-8485-4fee-875a-05d5327bfba4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=718f8ac6458a47f18440c145482ca9f8, user_id=709bda2cecbb40318182b0db9d7a0901, role_id=c4bf019fc6c44c71a6c984f6d0bab75f)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:33.219688 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74978779-8485-4fee-875a-05d5327bfba4 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:33.230799 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da803800-91fa-4909-b219-12135b2edc7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:33.231720 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da803800-91fa-4909-b219-12135b2edc7e admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:33.232175 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da803800-91fa-4909-b219-12135b2edc7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:33.232622 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da803800-91fa-4909-b219-12135b2edc7e admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:33.233272 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da803800-91fa-4909-b219-12135b2edc7e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:33.234910 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da803800-91fa-4909-b219-12135b2edc7e admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:33.281762 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 709/1428] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Nov 2 07:29:33 2022] PUT /identity/v3/projects/718f8ac6458a47f18440c145482ca9f8/users/709bda2cecbb40318182b0db9d7a0901/roles/c4bf019fc6c44c71a6c984f6d0bab75f => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:29:33.285182 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b556c89c-3440-49bf-89c9-4d7cbb09d0d2 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:33.285479 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b556c89c-3440-49bf-89c9-4d7cbb09d0d2 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:33.286550 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b556c89c-3440-49bf-89c9-4d7cbb09d0d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:33.294617 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 720/1429] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:29:33 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:29:33.295609 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da803800-91fa-4909-b219-12135b2edc7e admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:33.327752 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da803800-91fa-4909-b219-12135b2edc7e admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:33.369648 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-b556c89c-3440-49bf-89c9-4d7cbb09d0d2 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:29:33.374859 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-b556c89c-3440-49bf-89c9-4d7cbb09d0d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:33.548626 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:29:33.560330 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 710/1430] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:29:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:29:33.560330 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ee1926-10ea-4d67-99f3-900a3f7ab2b1 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:33.560330 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ee1926-10ea-4d67-99f3-900a3f7ab2b1 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:33.560330 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ee1926-10ea-4d67-99f3-900a3f7ab2b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:33.598971 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a96be0-54b9-499d-be3d-8d776b213187 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:33.599663 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a96be0-54b9-499d-be3d-8d776b213187 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:33.600006 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a96be0-54b9-499d-be3d-8d776b213187 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:33.600340 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a96be0-54b9-499d-be3d-8d776b213187 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:33.601841 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a96be0-54b9-499d-be3d-8d776b213187 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:33.605410 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a96be0-54b9-499d-be3d-8d776b213187 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:33.621411 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-d7ee1926-10ea-4d67-99f3-900a3f7ab2b1 None None] MFA Rules not processed for user `ccf803f8277740dca79913ee2f8b462b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:29:33.624439 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-d7ee1926-10ea-4d67-99f3-900a3f7ab2b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:33.625914 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 721/1431] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:29:33 2022] GET /identity/v3/roles => generated 1420 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:29:33.631486 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a96be0-54b9-499d-be3d-8d776b213187 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:33.661704 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a96be0-54b9-499d-be3d-8d776b213187 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:33.717825 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9593afd-61c2-4ab3-85a2-2b40b6393492 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:33.720752 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9593afd-61c2-4ab3-85a2-2b40b6393492 admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:33.723254 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9593afd-61c2-4ab3-85a2-2b40b6393492 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:33.724222 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9593afd-61c2-4ab3-85a2-2b40b6393492 admin admin] PATH_INFO: `/v3/projects/1b705ff84b404e42a998eddc878928b0/users/35e1d6073fb543fb97398c6ec63fda53/roles/b494fedd00f24f0b84a529e8672a156a` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:33.775894 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9593afd-61c2-4ab3-85a2-2b40b6393492 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1b705ff84b404e42a998eddc878928b0, user_id=35e1d6073fb543fb97398c6ec63fda53, role_id=b494fedd00f24f0b84a529e8672a156a)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:33.785778 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9593afd-61c2-4ab3-85a2-2b40b6393492 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:33.833173 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 722/1432] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Nov 2 07:29:33 2022] PUT /identity/v3/projects/1b705ff84b404e42a998eddc878928b0/users/35e1d6073fb543fb97398c6ec63fda53/roles/b494fedd00f24f0b84a529e8672a156a => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:29:33.833173 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9593afd-61c2-4ab3-85a2-2b40b6393492 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:33.833630 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:29:33.862649 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 711/1433] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:29:33 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 305 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:29:33.864279 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ee1926-10ea-4d67-99f3-900a3f7ab2b1 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:33.890803 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ee1926-10ea-4d67-99f3-900a3f7ab2b1 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:33.971825 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9593afd-61c2-4ab3-85a2-2b40b6393492 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:33.984755 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c99602-214d-4bbd-92e6-2f9a8cdd7a39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:33.985886 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c99602-214d-4bbd-92e6-2f9a8cdd7a39 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:33.985886 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c99602-214d-4bbd-92e6-2f9a8cdd7a39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:33.987587 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c99602-214d-4bbd-92e6-2f9a8cdd7a39 admin admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:33.989108 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c99602-214d-4bbd-92e6-2f9a8cdd7a39 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:29:33.989343 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c99602-214d-4bbd-92e6-2f9a8cdd7a39 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:33.993310 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c99602-214d-4bbd-92e6-2f9a8cdd7a39 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:34.027895 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8113292-655c-4805-9a14-c4bec1c6eb87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:34.028760 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8113292-655c-4805-9a14-c4bec1c6eb87 admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:34.031106 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8113292-655c-4805-9a14-c4bec1c6eb87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:34.031484 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8113292-655c-4805-9a14-c4bec1c6eb87 admin admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:34.032658 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8113292-655c-4805-9a14-c4bec1c6eb87 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:34.034812 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 723/1434] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Nov 2 07:29:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:29:34.036744 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8113292-655c-4805-9a14-c4bec1c6eb87 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:34.037641 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df85142a-080c-45bb-b437-6703bb6e8a4f None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:34.037948 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df85142a-080c-45bb-b437-6703bb6e8a4f None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:34.038278 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df85142a-080c-45bb-b437-6703bb6e8a4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:34.063968 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 712/1435] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:29:33 2022] GET /identity/v3/roles => generated 1420 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:29:34.066690 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fe35ee-9db6-4297-935c-b5c105248cb3 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:34.067002 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fe35ee-9db6-4297-935c-b5c105248cb3 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:34.067277 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fe35ee-9db6-4297-935c-b5c105248cb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:34.113474 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-33fe35ee-9db6-4297-935c-b5c105248cb3 None None] MFA Rules not processed for user `35e1d6073fb543fb97398c6ec63fda53`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:29:34.117402 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-33fe35ee-9db6-4297-935c-b5c105248cb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:34.179695 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-df85142a-080c-45bb-b437-6703bb6e8a4f None None] MFA Rules not processed for user `ff4736f3d50746b0bf5d6a917b2ca0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:29:34.180885 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-df85142a-080c-45bb-b437-6703bb6e8a4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:34.266845 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 713/1436] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:29:34 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:29:34.267778 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fe35ee-9db6-4297-935c-b5c105248cb3 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:34.269653 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-33fe35ee-9db6-4297-935c-b5c105248cb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:34.346492 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 724/1437] 10.222.0.18 () {58 vars in 1009 bytes} [Wed Nov 2 07:29:34 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 311 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:29:34.347339 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df85142a-080c-45bb-b437-6703bb6e8a4f None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:34.381136 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df85142a-080c-45bb-b437-6703bb6e8a4f None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:34.384547 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fe35ee-9db6-4297-935c-b5c105248cb3 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:34.390734 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ca87e7-f496-44d1-8a0b-9b70fd90e31b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43bb326fb82e4d2c83fb2eaf91860dc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:34.392179 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ca87e7-f496-44d1-8a0b-9b70fd90e31b service cinder] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:34.392179 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ca87e7-f496-44d1-8a0b-9b70fd90e31b service cinder] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:34.392674 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ca87e7-f496-44d1-8a0b-9b70fd90e31b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:34.406430 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f257bf75-438d-41f6-98e0-8c5b2197bc69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:34.407088 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f257bf75-438d-41f6-98e0-8c5b2197bc69 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:34.407341 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f257bf75-438d-41f6-98e0-8c5b2197bc69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:34.407615 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f257bf75-438d-41f6-98e0-8c5b2197bc69 admin admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:34.408266 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f257bf75-438d-41f6-98e0-8c5b2197bc69 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:34.410190 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f257bf75-438d-41f6-98e0-8c5b2197bc69 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:34.435342 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:29:34.473922 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:29:34.499476 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 725/1438] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Nov 2 07:29:34 2022] POST /identity/v3/projects => generated 373 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:29:34.499476 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f257bf75-438d-41f6-98e0-8c5b2197bc69 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:34.523732 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ca87e7-f496-44d1-8a0b-9b70fd90e31b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:34.536990 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ca87e7-f496-44d1-8a0b-9b70fd90e31b service cinder] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:34.592527 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 714/1439] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:29:34 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:29:34.593287 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ca87e7-f496-44d1-8a0b-9b70fd90e31b service cinder] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:34.653515 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ca87e7-f496-44d1-8a0b-9b70fd90e31b service cinder] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:34.872102 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f257bf75-438d-41f6-98e0-8c5b2197bc69 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:34.890976 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b22145-f656-4197-a8c5-fb8c63d6b487 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:34.892480 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b22145-f656-4197-a8c5-fb8c63d6b487 admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:34.897027 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b22145-f656-4197-a8c5-fb8c63d6b487 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:34.897357 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b22145-f656-4197-a8c5-fb8c63d6b487 admin admin] PATH_INFO: `/v3/projects/5f7250b8cb574e96a41dd8123062dc3f/users/603d63b3db5e4363b449974b99a56a44/roles/b494fedd00f24f0b84a529e8672a156a` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:34.922785 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471edf0a-7de4-4209-866a-6062d5193c60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:34.932454 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471edf0a-7de4-4209-866a-6062d5193c60 admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:34.934087 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471edf0a-7de4-4209-866a-6062d5193c60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:34.935091 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471edf0a-7de4-4209-866a-6062d5193c60 admin admin] PATH_INFO: `/v3/projects` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:34.936544 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471edf0a-7de4-4209-866a-6062d5193c60 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:34.939309 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471edf0a-7de4-4209-866a-6062d5193c60 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:34.967440 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b22145-f656-4197-a8c5-fb8c63d6b487 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f7250b8cb574e96a41dd8123062dc3f, user_id=603d63b3db5e4363b449974b99a56a44, role_id=b494fedd00f24f0b84a529e8672a156a)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:34.970141 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b22145-f656-4197-a8c5-fb8c63d6b487 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:34.994295 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 715/1440] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Nov 2 07:29:34 2022] POST /identity/v3/projects => generated 373 bytes in 401 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:29:35.007627 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcee604-29ef-4a37-bad0-592569b1c8bd None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:35.008320 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcee604-29ef-4a37-bad0-592569b1c8bd None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:35.009344 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 726/1441] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Nov 2 07:29:34 2022] PUT /identity/v3/projects/5f7250b8cb574e96a41dd8123062dc3f/users/603d63b3db5e4363b449974b99a56a44/roles/b494fedd00f24f0b84a529e8672a156a => generated 0 bytes in 513 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:29:35.009853 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcee604-29ef-4a37-bad0-592569b1c8bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:35.019913 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b22145-f656-4197-a8c5-fb8c63d6b487 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:35.082703 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-0dcee604-29ef-4a37-bad0-592569b1c8bd None None] MFA Rules not processed for user `2455ebfc92ba4d398a6279d6102a0408`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:29:35.085082 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-0dcee604-29ef-4a37-bad0-592569b1c8bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:35.170217 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b22145-f656-4197-a8c5-fb8c63d6b487 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:35.170379 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 716/1442] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:29:34 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:29:35.171439 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcee604-29ef-4a37-bad0-592569b1c8bd None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:35.173026 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-0dcee604-29ef-4a37-bad0-592569b1c8bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:35.186102 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ecec01-9ebb-4516-8651-bdc893839008 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:35.186775 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ecec01-9ebb-4516-8651-bdc893839008 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:35.187056 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ecec01-9ebb-4516-8651-bdc893839008 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:35.187325 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ecec01-9ebb-4516-8651-bdc893839008 admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:35.187959 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ecec01-9ebb-4516-8651-bdc893839008 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:35.190119 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ecec01-9ebb-4516-8651-bdc893839008 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:35.249511 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 727/1443] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Nov 2 07:29:35 2022] POST /identity/v3/users => generated 287 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:29:35.254906 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ecec01-9ebb-4516-8651-bdc893839008 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:35.412365 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:29:35.463739 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ecec01-9ebb-4516-8651-bdc893839008 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:35.470239 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54eb6507-91a2-4534-8ca9-a4be1a52002a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43bb326fb82e4d2c83fb2eaf91860dc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:35.471887 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54eb6507-91a2-4534-8ca9-a4be1a52002a service cinder] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:35.472382 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54eb6507-91a2-4534-8ca9-a4be1a52002a service cinder] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:35.473158 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54eb6507-91a2-4534-8ca9-a4be1a52002a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:35.476439 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-54eb6507-91a2-4534-8ca9-a4be1a52002a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:35.496077 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcee604-29ef-4a37-bad0-592569b1c8bd None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:35.512667 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bbb48e-b689-4776-95e7-5b2a54afd68d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9116f47889f4db39e0203d6cdae1593', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:35.513310 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bbb48e-b689-4776-95e7-5b2a54afd68d service swift] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:35.513591 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bbb48e-b689-4776-95e7-5b2a54afd68d service swift] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:35.513867 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bbb48e-b689-4776-95e7-5b2a54afd68d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:35.518593 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:29:35.597944 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54eb6507-91a2-4534-8ca9-a4be1a52002a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:35.599463 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54eb6507-91a2-4534-8ca9-a4be1a52002a service cinder] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:35.621046 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 728/1444] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:29:35 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:29:35.622163 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54eb6507-91a2-4534-8ca9-a4be1a52002a service cinder] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:35.637687 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54eb6507-91a2-4534-8ca9-a4be1a52002a service cinder] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:35.656689 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9161d2d8-2324-48b4-bd7f-080444ea49dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:35.657432 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9161d2d8-2324-48b4-bd7f-080444ea49dc admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:35.660120 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9161d2d8-2324-48b4-bd7f-080444ea49dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:35.660524 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9161d2d8-2324-48b4-bd7f-080444ea49dc admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:35.661726 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bbb48e-b689-4776-95e7-5b2a54afd68d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:35.664595 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bbb48e-b689-4776-95e7-5b2a54afd68d service swift] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:35.665257 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9161d2d8-2324-48b4-bd7f-080444ea49dc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:35.667522 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9161d2d8-2324-48b4-bd7f-080444ea49dc admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:35.711153 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 717/1445] 10.222.0.18 () {60 vars in 1384 bytes} [Wed Nov 2 07:29:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 540 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Nov 02 07:29:35.715098 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c45595-1ba8-47bb-a3d8-25121139c1df None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:35.715641 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c45595-1ba8-47bb-a3d8-25121139c1df None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:35.717159 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c45595-1ba8-47bb-a3d8-25121139c1df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:35.743310 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 729/1446] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Nov 2 07:29:35 2022] POST /identity/v3/users => generated 293 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:29:35.744568 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9161d2d8-2324-48b4-bd7f-080444ea49dc admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:35.779500 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9161d2d8-2324-48b4-bd7f-080444ea49dc admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:35.794112 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d01b47-a4cb-4e9a-b7aa-57ca90ee489f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:35.794934 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d01b47-a4cb-4e9a-b7aa-57ca90ee489f admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:35.797325 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d01b47-a4cb-4e9a-b7aa-57ca90ee489f admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:35.797325 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d01b47-a4cb-4e9a-b7aa-57ca90ee489f admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:35.800721 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d01b47-a4cb-4e9a-b7aa-57ca90ee489f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:35.805754 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d01b47-a4cb-4e9a-b7aa-57ca90ee489f admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:35.844909 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 730/1447] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:29:35 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:29:35.846387 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71438923-68b6-4bee-b3de-d0dca3c1344e None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:35.846685 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71438923-68b6-4bee-b3de-d0dca3c1344e None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:35.846964 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71438923-68b6-4bee-b3de-d0dca3c1344e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:35.867861 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-78c45595-1ba8-47bb-a3d8-25121139c1df None None] MFA Rules not processed for user `603d63b3db5e4363b449974b99a56a44`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:29:35.868982 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-78c45595-1ba8-47bb-a3d8-25121139c1df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:35.921427 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-71438923-68b6-4bee-b3de-d0dca3c1344e None None] MFA Rules not processed for user `b3ec6ecb85f94ef59762877d9ea33ed0`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:29:35.923492 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-71438923-68b6-4bee-b3de-d0dca3c1344e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:36.123376 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 731/1448] 10.222.0.47 () {58 vars in 1015 bytes} [Wed Nov 2 07:29:35 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:29:36.124585 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71438923-68b6-4bee-b3de-d0dca3c1344e None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:36.140437 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 718/1449] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:29:35 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 429 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:29:36.141629 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c45595-1ba8-47bb-a3d8-25121139c1df None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:36.163080 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71438923-68b6-4bee-b3de-d0dca3c1344e None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:36.171160 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c45595-1ba8-47bb-a3d8-25121139c1df None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:36.183996 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0558e5-8308-4911-9524-a9a859dbb03c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:36.184897 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0558e5-8308-4911-9524-a9a859dbb03c admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:36.185346 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0558e5-8308-4911-9524-a9a859dbb03c admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:36.185799 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0558e5-8308-4911-9524-a9a859dbb03c admin admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:36.185917 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1492aec-41a9-42ee-ba04-4851d10f84d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:36.186524 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1492aec-41a9-42ee-ba04-4851d10f84d5 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:36.186866 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1492aec-41a9-42ee-ba04-4851d10f84d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:36.187142 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1492aec-41a9-42ee-ba04-4851d10f84d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:36.187542 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0558e5-8308-4911-9524-a9a859dbb03c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:36.188327 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1492aec-41a9-42ee-ba04-4851d10f84d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:36.192854 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1492aec-41a9-42ee-ba04-4851d10f84d5 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:36.193750 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0558e5-8308-4911-9524-a9a859dbb03c admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:36.228872 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 719/1450] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:29:36 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:29:36.229604 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0558e5-8308-4911-9524-a9a859dbb03c admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:36.233863 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 732/1451] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:29:36 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:29:36.239445 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1492aec-41a9-42ee-ba04-4851d10f84d5 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:36.248226 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0558e5-8308-4911-9524-a9a859dbb03c admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:36.253764 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd08e8e-ace8-484a-892f-b607b4992500 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:36.254667 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd08e8e-ace8-484a-892f-b607b4992500 admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:36.255017 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd08e8e-ace8-484a-892f-b607b4992500 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:36.255354 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd08e8e-ace8-484a-892f-b607b4992500 admin admin] PATH_INFO: `/v3/projects` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:36.256070 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd08e8e-ace8-484a-892f-b607b4992500 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:36.258055 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd08e8e-ace8-484a-892f-b607b4992500 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:36.281418 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1492aec-41a9-42ee-ba04-4851d10f84d5 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:36.307656 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891376bc-d404-4021-bc89-c411129569d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:36.308877 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891376bc-d404-4021-bc89-c411129569d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:36.309308 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891376bc-d404-4021-bc89-c411129569d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:36.310614 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891376bc-d404-4021-bc89-c411129569d6 admin admin] PATH_INFO: `/v3/projects/d067b8b0214c4e7d8141df0bac1f945e/users/f4d23fe9a786498b9cf7e00cb5b983a8/roles/b494fedd00f24f0b84a529e8672a156a` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:36.321741 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 720/1452] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Nov 2 07:29:36 2022] POST /identity/v3/projects => generated 377 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:29:36.322875 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd08e8e-ace8-484a-892f-b607b4992500 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:36.374678 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891376bc-d404-4021-bc89-c411129569d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d067b8b0214c4e7d8141df0bac1f945e, user_id=f4d23fe9a786498b9cf7e00cb5b983a8, role_id=b494fedd00f24f0b84a529e8672a156a)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:36.379504 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891376bc-d404-4021-bc89-c411129569d6 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:36.429627 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:29:36.437973 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 733/1453] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Nov 2 07:29:36 2022] PUT /identity/v3/projects/d067b8b0214c4e7d8141df0bac1f945e/users/f4d23fe9a786498b9cf7e00cb5b983a8/roles/b494fedd00f24f0b84a529e8672a156a => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:29:36.455762 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891376bc-d404-4021-bc89-c411129569d6 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:36.507798 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd08e8e-ace8-484a-892f-b607b4992500 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:36.695594 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891376bc-d404-4021-bc89-c411129569d6 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:36.709160 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39dc7d0b-8589-45d2-a169-5943b4b08997 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:36.711743 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dc7d0b-8589-45d2-a169-5943b4b08997 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:36.713013 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dc7d0b-8589-45d2-a169-5943b4b08997 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:36.715210 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dc7d0b-8589-45d2-a169-5943b4b08997 admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:36.719483 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39dc7d0b-8589-45d2-a169-5943b4b08997 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:36.725630 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5fbb1b-15da-4b9e-bd19-cf62c359095a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:36.726637 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5fbb1b-15da-4b9e-bd19-cf62c359095a admin admin] REQUEST_METHOD: `PUT` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:36.726918 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5fbb1b-15da-4b9e-bd19-cf62c359095a admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:36.727480 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5fbb1b-15da-4b9e-bd19-cf62c359095a admin admin] PATH_INFO: `/v3/projects/6965bb3ff72d43c0a8dfcf873424656f/users/459d8e2afac14ea780ebd0cf615a88f2/roles/c4bf019fc6c44c71a6c984f6d0bab75f` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:36.736141 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39dc7d0b-8589-45d2-a169-5943b4b08997 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:36.766231 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5fbb1b-15da-4b9e-bd19-cf62c359095a admin admin] RBAC: Authorizing `identity:create_grant(project_id=6965bb3ff72d43c0a8dfcf873424656f, user_id=459d8e2afac14ea780ebd0cf615a88f2, role_id=c4bf019fc6c44c71a6c984f6d0bab75f)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:36.778036 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5fbb1b-15da-4b9e-bd19-cf62c359095a admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:36.812911 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 721/1454] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Nov 2 07:29:36 2022] PUT /identity/v3/projects/6965bb3ff72d43c0a8dfcf873424656f/users/459d8e2afac14ea780ebd0cf615a88f2/roles/c4bf019fc6c44c71a6c984f6d0bab75f => generated 0 bytes in 487 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:29:36.813782 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5fbb1b-15da-4b9e-bd19-cf62c359095a admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:36.818721 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-5c5fbb1b-15da-4b9e-bd19-cf62c359095a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:36.819626 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:29:36.855782 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 734/1455] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Nov 2 07:29:36 2022] POST /identity/v3/users => generated 295 bytes in 417 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:29:36.858623 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297cc540-5545-4235-8308-8c705cfcbfdc None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:36.862316 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297cc540-5545-4235-8308-8c705cfcbfdc None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:36.863789 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297cc540-5545-4235-8308-8c705cfcbfdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:36.959857 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5fbb1b-15da-4b9e-bd19-cf62c359095a admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:36.971433 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7f5dd1-cd90-44e1-b07c-2cb46c64c6b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3ec6ecb85f94ef59762877d9ea33ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:36.972138 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7f5dd1-cd90-44e1-b07c-2cb46c64c6b8 service neutron] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:36.972571 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7f5dd1-cd90-44e1-b07c-2cb46c64c6b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:36.972895 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7f5dd1-cd90-44e1-b07c-2cb46c64c6b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:36.994310 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7f5dd1-cd90-44e1-b07c-2cb46c64c6b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:37.009867 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7f5dd1-cd90-44e1-b07c-2cb46c64c6b8 service neutron] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:37.039676 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 722/1456] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:29:36 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:29:37.045361 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7f5dd1-cd90-44e1-b07c-2cb46c64c6b8 service neutron] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:37.048568 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-297cc540-5545-4235-8308-8c705cfcbfdc None None] MFA Rules not processed for user `f4d23fe9a786498b9cf7e00cb5b983a8`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:29:37.055458 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-297cc540-5545-4235-8308-8c705cfcbfdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:37.207212 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7f5dd1-cd90-44e1-b07c-2cb46c64c6b8 service neutron] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:37.218285 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3420bff-2ebc-4bdf-a5af-1baa8cd6001a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:37.220140 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3420bff-2ebc-4bdf-a5af-1baa8cd6001a admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:37.221324 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3420bff-2ebc-4bdf-a5af-1baa8cd6001a admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:37.221662 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3420bff-2ebc-4bdf-a5af-1baa8cd6001a admin admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:37.222410 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3420bff-2ebc-4bdf-a5af-1baa8cd6001a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:37.231062 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3420bff-2ebc-4bdf-a5af-1baa8cd6001a admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:37.274818 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 723/1457] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:29:37 2022] GET /identity/v3/roles => generated 1420 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:29:37.279211 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3420bff-2ebc-4bdf-a5af-1baa8cd6001a admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:37.295722 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3420bff-2ebc-4bdf-a5af-1baa8cd6001a admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:37.303203 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f346e7b-5107-4d6d-82f0-a28141023e8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:37.303883 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f346e7b-5107-4d6d-82f0-a28141023e8b admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:37.304173 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f346e7b-5107-4d6d-82f0-a28141023e8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:37.305576 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f346e7b-5107-4d6d-82f0-a28141023e8b admin admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:37.306246 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f346e7b-5107-4d6d-82f0-a28141023e8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:37.313096 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f346e7b-5107-4d6d-82f0-a28141023e8b admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:37.319099 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:29:37.333509 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 735/1458] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:29:36 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 477 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:29:37.336821 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297cc540-5545-4235-8308-8c705cfcbfdc None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:37.349040 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 724/1459] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:29:37 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:29:37.349994 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f346e7b-5107-4d6d-82f0-a28141023e8b admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:37.403047 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297cc540-5545-4235-8308-8c705cfcbfdc None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:37.424265 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca87f89f-b07b-4295-811a-1d179bb495da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:37.425422 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca87f89f-b07b-4295-811a-1d179bb495da admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:37.425422 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca87f89f-b07b-4295-811a-1d179bb495da admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:37.425572 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca87f89f-b07b-4295-811a-1d179bb495da admin admin] PATH_INFO: `/v3/projects/6965bb3ff72d43c0a8dfcf873424656f/users/459d8e2afac14ea780ebd0cf615a88f2/roles/c4bf019fc6c44c71a6c984f6d0bab75f` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:37.426130 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f346e7b-5107-4d6d-82f0-a28141023e8b admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:37.434495 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca87f89f-b07b-4295-811a-1d179bb495da admin admin] RBAC: Authorizing `identity:create_grant(project_id=6965bb3ff72d43c0a8dfcf873424656f, user_id=459d8e2afac14ea780ebd0cf615a88f2, role_id=c4bf019fc6c44c71a6c984f6d0bab75f)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:37.456682 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca87f89f-b07b-4295-811a-1d179bb495da admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:37.457058 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9c63a0-8923-4d78-a21b-de82be43a03f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:37.457753 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9c63a0-8923-4d78-a21b-de82be43a03f admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:37.458041 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9c63a0-8923-4d78-a21b-de82be43a03f admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:37.458317 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9c63a0-8923-4d78-a21b-de82be43a03f admin admin] PATH_INFO: `/v3/projects` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:37.458991 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9c63a0-8923-4d78-a21b-de82be43a03f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:37.471302 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9c63a0-8923-4d78-a21b-de82be43a03f admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:37.523511 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 736/1460] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Nov 2 07:29:37 2022] PUT /identity/v3/projects/6965bb3ff72d43c0a8dfcf873424656f/users/459d8e2afac14ea780ebd0cf615a88f2/roles/c4bf019fc6c44c71a6c984f6d0bab75f => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:29:37.524577 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca87f89f-b07b-4295-811a-1d179bb495da admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:37.591737 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 725/1461] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Nov 2 07:29:37 2022] POST /identity/v3/projects => generated 373 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:29:37.593367 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5888ee-8a4a-4023-9919-851737e34fc7 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:37.593812 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5888ee-8a4a-4023-9919-851737e34fc7 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:37.594233 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5888ee-8a4a-4023-9919-851737e34fc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:37.777622 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-4b5888ee-8a4a-4023-9919-851737e34fc7 None None] MFA Rules not processed for user `459d8e2afac14ea780ebd0cf615a88f2`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:29:37.778884 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-4b5888ee-8a4a-4023-9919-851737e34fc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:37.804307 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca87f89f-b07b-4295-811a-1d179bb495da admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:38.062342 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2325a3bb-9d74-44e3-b9bc-8af5e300a6c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:38.066285 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2325a3bb-9d74-44e3-b9bc-8af5e300a6c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:38.066612 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2325a3bb-9d74-44e3-b9bc-8af5e300a6c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:38.066879 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2325a3bb-9d74-44e3-b9bc-8af5e300a6c7 admin admin] PATH_INFO: `/v3/projects/011a28d25afe452dac9be134650a1345/users/359f0d29a38145419dc8d571f7f17a08/roles/c4bf019fc6c44c71a6c984f6d0bab75f` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:38.102181 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 726/1462] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:29:37 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 510 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:29:38.103212 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5888ee-8a4a-4023-9919-851737e34fc7 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:38.172602 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5888ee-8a4a-4023-9919-851737e34fc7 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:38.212081 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9899b9-1fcc-48ae-98f3-01b009ae98ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:38.212720 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9899b9-1fcc-48ae-98f3-01b009ae98ac admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:38.213006 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9899b9-1fcc-48ae-98f3-01b009ae98ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:38.213276 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9899b9-1fcc-48ae-98f3-01b009ae98ac admin admin] PATH_INFO: `/v3/users` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:38.213978 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9899b9-1fcc-48ae-98f3-01b009ae98ac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:38.216795 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9899b9-1fcc-48ae-98f3-01b009ae98ac admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:38.295147 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 727/1463] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Nov 2 07:29:38 2022] POST /identity/v3/users => generated 293 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:29:38.298376 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e3f474-37d8-48e8-8b0a-a6d3f7841090 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:38.299091 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e3f474-37d8-48e8-8b0a-a6d3f7841090 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:38.299195 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e3f474-37d8-48e8-8b0a-a6d3f7841090 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:38.360184 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-37e3f474-37d8-48e8-8b0a-a6d3f7841090 None None] MFA Rules not processed for user `709bda2cecbb40318182b0db9d7a0901`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:29:38.365210 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-37e3f474-37d8-48e8-8b0a-a6d3f7841090 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:38.592883 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 728/1464] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:29:38 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 297 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:29:38.594241 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e3f474-37d8-48e8-8b0a-a6d3f7841090 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:38.596279 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2325a3bb-9d74-44e3-b9bc-8af5e300a6c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=011a28d25afe452dac9be134650a1345, user_id=359f0d29a38145419dc8d571f7f17a08, role_id=c4bf019fc6c44c71a6c984f6d0bab75f)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:38.611529 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2325a3bb-9d74-44e3-b9bc-8af5e300a6c7 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:38.675772 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:29:38.688474 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 737/1465] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Nov 2 07:29:37 2022] PUT /identity/v3/projects/011a28d25afe452dac9be134650a1345/users/359f0d29a38145419dc8d571f7f17a08/roles/c4bf019fc6c44c71a6c984f6d0bab75f => generated 0 bytes in 1162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:29:38.698420 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2325a3bb-9d74-44e3-b9bc-8af5e300a6c7 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:38.734255 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e3f474-37d8-48e8-8b0a-a6d3f7841090 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:38.753653 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:29:38.850132 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d0e77e-a869-464c-aa3c-20a204363013 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43bb326fb82e4d2c83fb2eaf91860dc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:38.850949 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d0e77e-a869-464c-aa3c-20a204363013 service cinder] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:38.851608 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d0e77e-a869-464c-aa3c-20a204363013 service cinder] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:38.852094 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d0e77e-a869-464c-aa3c-20a204363013 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:38.898239 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2325a3bb-9d74-44e3-b9bc-8af5e300a6c7 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:38.915402 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254ed90d-9228-4f8c-ab92-020d8c7d9b43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:38.916142 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254ed90d-9228-4f8c-ab92-020d8c7d9b43 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:38.917965 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254ed90d-9228-4f8c-ab92-020d8c7d9b43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:38.918306 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254ed90d-9228-4f8c-ab92-020d8c7d9b43 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:38.921549 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254ed90d-9228-4f8c-ab92-020d8c7d9b43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:38.925959 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254ed90d-9228-4f8c-ab92-020d8c7d9b43 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:38.983022 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 738/1466] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:29:38 2022] GET /identity/v3/roles => generated 1420 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:29:38.983022 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254ed90d-9228-4f8c-ab92-020d8c7d9b43 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:39.019893 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254ed90d-9228-4f8c-ab92-020d8c7d9b43 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:39.035298 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8a8ceb-cba9-48e9-9cee-83cf269995f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:39.036118 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8a8ceb-cba9-48e9-9cee-83cf269995f7 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:39.036403 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8a8ceb-cba9-48e9-9cee-83cf269995f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:39.037888 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8a8ceb-cba9-48e9-9cee-83cf269995f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:39.039714 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8a8ceb-cba9-48e9-9cee-83cf269995f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:39.044869 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8a8ceb-cba9-48e9-9cee-83cf269995f7 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:39.061665 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d0e77e-a869-464c-aa3c-20a204363013 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:39.064257 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d0e77e-a869-464c-aa3c-20a204363013 service cinder] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:39.078199 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 739/1467] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:29:38 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:29:39.079137 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8a8ceb-cba9-48e9-9cee-83cf269995f7 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:39.093042 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 729/1468] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:29:38 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 500 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:29:39.093978 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d0e77e-a869-464c-aa3c-20a204363013 service cinder] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:39.113060 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8a8ceb-cba9-48e9-9cee-83cf269995f7 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:39.129588 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d0e77e-a869-464c-aa3c-20a204363013 service cinder] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:39.140538 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc26833-d6b2-458e-ab9b-b1a902aa269e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:39.141846 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc26833-d6b2-458e-ab9b-b1a902aa269e admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:39.142308 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc26833-d6b2-458e-ab9b-b1a902aa269e admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:39.142855 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc26833-d6b2-458e-ab9b-b1a902aa269e admin admin] PATH_INFO: `/v3/projects/a42f5fc3552c4de683f3d2260e83bcba/users/f1a726917d034f6db1743bc7ac4be4d5/roles/c4bf019fc6c44c71a6c984f6d0bab75f` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:39.163263 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fef286-2f55-4f6e-a5ac-77e6c78ae5f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:39.164080 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fef286-2f55-4f6e-a5ac-77e6c78ae5f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:39.164527 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fef286-2f55-4f6e-a5ac-77e6c78ae5f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:39.166152 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fef286-2f55-4f6e-a5ac-77e6c78ae5f4 admin admin] PATH_INFO: `/v3/projects/011a28d25afe452dac9be134650a1345/users/359f0d29a38145419dc8d571f7f17a08/roles/c4bf019fc6c44c71a6c984f6d0bab75f` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:39.174101 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fef286-2f55-4f6e-a5ac-77e6c78ae5f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=011a28d25afe452dac9be134650a1345, user_id=359f0d29a38145419dc8d571f7f17a08, role_id=c4bf019fc6c44c71a6c984f6d0bab75f)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:39.178674 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fef286-2f55-4f6e-a5ac-77e6c78ae5f4 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:39.242069 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 730/1469] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Nov 2 07:29:39 2022] PUT /identity/v3/projects/011a28d25afe452dac9be134650a1345/users/359f0d29a38145419dc8d571f7f17a08/roles/c4bf019fc6c44c71a6c984f6d0bab75f => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:29:39.252429 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fef286-2f55-4f6e-a5ac-77e6c78ae5f4 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:39.260472 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc26833-d6b2-458e-ab9b-b1a902aa269e admin admin] RBAC: Authorizing `identity:create_grant(project_id=a42f5fc3552c4de683f3d2260e83bcba, user_id=f1a726917d034f6db1743bc7ac4be4d5, role_id=c4bf019fc6c44c71a6c984f6d0bab75f)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:39.263856 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc26833-d6b2-458e-ab9b-b1a902aa269e admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:39.301813 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 740/1470] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Nov 2 07:29:39 2022] PUT /identity/v3/projects/a42f5fc3552c4de683f3d2260e83bcba/users/f1a726917d034f6db1743bc7ac4be4d5/roles/c4bf019fc6c44c71a6c984f6d0bab75f => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:29:39.303339 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f635946f-b51b-4af3-aa8b-7069d1a6f480 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:39.303680 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f635946f-b51b-4af3-aa8b-7069d1a6f480 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:39.303905 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f635946f-b51b-4af3-aa8b-7069d1a6f480 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:39.402052 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-f635946f-b51b-4af3-aa8b-7069d1a6f480 None None] MFA Rules not processed for user `359f0d29a38145419dc8d571f7f17a08`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:29:39.408494 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-f635946f-b51b-4af3-aa8b-7069d1a6f480 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:39.484954 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fef286-2f55-4f6e-a5ac-77e6c78ae5f4 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:39.578627 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:29:39.607861 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 741/1471] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:29:39 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 305 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:29:39.621836 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f635946f-b51b-4af3-aa8b-7069d1a6f480 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:39.715223 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b835b2-a472-40cb-a113-afa8c61e0948 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:39.716069 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b835b2-a472-40cb-a113-afa8c61e0948 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:39.716438 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b835b2-a472-40cb-a113-afa8c61e0948 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:39.716795 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b835b2-a472-40cb-a113-afa8c61e0948 admin admin] PATH_INFO: `/v3/users/c774a9a98614453eadd676a7b51c72c7` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:39.720606 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b835b2-a472-40cb-a113-afa8c61e0948 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c774a9a98614453eadd676a7b51c72c7)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:39.737716 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b835b2-a472-40cb-a113-afa8c61e0948 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:39.820528 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:29:39.880485 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f635946f-b51b-4af3-aa8b-7069d1a6f480 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:39.900614 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc439df-3614-4b0e-8d89-0fe27d6a9dff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:39.901243 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc439df-3614-4b0e-8d89-0fe27d6a9dff admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:39.901514 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc439df-3614-4b0e-8d89-0fe27d6a9dff admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:39.902506 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc439df-3614-4b0e-8d89-0fe27d6a9dff admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:39.903241 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc439df-3614-4b0e-8d89-0fe27d6a9dff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:39.904849 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc439df-3614-4b0e-8d89-0fe27d6a9dff admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:39.922791 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-00b835b2-a472-40cb-a113-afa8c61e0948 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c774a9a98614453eadd676a7b51c72c7'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:39.953575 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-00b835b2-a472-40cb-a113-afa8c61e0948 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c774a9a98614453eadd676a7b51c72c7'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:39.960910 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-00b835b2-a472-40cb-a113-afa8c61e0948 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c774a9a98614453eadd676a7b51c72c7'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:39.990385 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-00b835b2-a472-40cb-a113-afa8c61e0948 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c774a9a98614453eadd676a7b51c72c7'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:39.990640 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 742/1472] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:29:39 2022] GET /identity/v3/roles => generated 1420 bytes in 376 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:29:39.990640 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc439df-3614-4b0e-8d89-0fe27d6a9dff admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:39.993611 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-4dc439df-3614-4b0e-8d89-0fe27d6a9dff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:40.024718 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-00b835b2-a472-40cb-a113-afa8c61e0948 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c774a9a98614453eadd676a7b51c72c7'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:40.046302 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 731/1473] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Nov 2 07:29:39 2022] DELETE /identity/v3/users/c774a9a98614453eadd676a7b51c72c7 => generated 0 bytes in 794 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:29:40.049526 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b835b2-a472-40cb-a113-afa8c61e0948 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:40.055714 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-00b835b2-a472-40cb-a113-afa8c61e0948 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:40.207359 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc439df-3614-4b0e-8d89-0fe27d6a9dff admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:40.282646 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b835b2-a472-40cb-a113-afa8c61e0948 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:40.294594 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd0d3de-3c3e-4c81-9d40-2014f699e6cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:40.296431 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd0d3de-3c3e-4c81-9d40-2014f699e6cd admin admin] REQUEST_METHOD: `PUT` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:40.297584 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd0d3de-3c3e-4c81-9d40-2014f699e6cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:40.298028 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd0d3de-3c3e-4c81-9d40-2014f699e6cd admin admin] PATH_INFO: `/v3/projects/a42f5fc3552c4de683f3d2260e83bcba/users/f1a726917d034f6db1743bc7ac4be4d5/roles/c4bf019fc6c44c71a6c984f6d0bab75f` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:40.305752 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd0d3de-3c3e-4c81-9d40-2014f699e6cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=a42f5fc3552c4de683f3d2260e83bcba, user_id=f1a726917d034f6db1743bc7ac4be4d5, role_id=c4bf019fc6c44c71a6c984f6d0bab75f)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:40.313906 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd0d3de-3c3e-4c81-9d40-2014f699e6cd admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:40.359902 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 732/1474] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Nov 2 07:29:40 2022] PUT /identity/v3/projects/a42f5fc3552c4de683f3d2260e83bcba/users/f1a726917d034f6db1743bc7ac4be4d5/roles/c4bf019fc6c44c71a6c984f6d0bab75f => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:29:40.364077 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c118a3f-1539-4fb8-9b46-13b4e3a69ca0 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:40.364553 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c118a3f-1539-4fb8-9b46-13b4e3a69ca0 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:40.366235 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c118a3f-1539-4fb8-9b46-13b4e3a69ca0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:40.419752 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-3c118a3f-1539-4fb8-9b46-13b4e3a69ca0 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:29:40.424615 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-3c118a3f-1539-4fb8-9b46-13b4e3a69ca0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:40.522276 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:29:40.533017 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb8eaa2-e5c4-4407-bc65-cddcd3cf4807 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43bb326fb82e4d2c83fb2eaf91860dc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:40.535609 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb8eaa2-e5c4-4407-bc65-cddcd3cf4807 service cinder] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:40.535903 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb8eaa2-e5c4-4407-bc65-cddcd3cf4807 service cinder] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:40.538811 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb8eaa2-e5c4-4407-bc65-cddcd3cf4807 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:40.544673 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-edb8eaa2-e5c4-4407-bc65-cddcd3cf4807 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:40.622913 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 733/1475] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:29:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:29:40.627151 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d6961d-1afd-4f5b-81c9-4781e48e2903 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:40.627570 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d6961d-1afd-4f5b-81c9-4781e48e2903 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:40.627817 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d6961d-1afd-4f5b-81c9-4781e48e2903 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:40.721169 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-09d6961d-1afd-4f5b-81c9-4781e48e2903 None None] MFA Rules not processed for user `f1a726917d034f6db1743bc7ac4be4d5`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:29:40.724130 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-09d6961d-1afd-4f5b-81c9-4781e48e2903 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:40.748546 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb8eaa2-e5c4-4407-bc65-cddcd3cf4807 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:40.750101 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb8eaa2-e5c4-4407-bc65-cddcd3cf4807 service cinder] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:40.780233 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 743/1476] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:29:39 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 795 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:29:40.781101 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb8eaa2-e5c4-4407-bc65-cddcd3cf4807 service cinder] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:40.796299 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-edb8eaa2-e5c4-4407-bc65-cddcd3cf4807 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:40.831562 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb8eaa2-e5c4-4407-bc65-cddcd3cf4807 service cinder] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:40.846907 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c30b650-4831-422b-979b-4dbc6efb830a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:40.848792 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c30b650-4831-422b-979b-4dbc6efb830a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:40.849269 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c30b650-4831-422b-979b-4dbc6efb830a admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:40.850410 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c30b650-4831-422b-979b-4dbc6efb830a admin admin] PATH_INFO: `/v3/users/35e1d6073fb543fb97398c6ec63fda53` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:40.856767 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c30b650-4831-422b-979b-4dbc6efb830a admin admin] RBAC: Authorizing `identity:delete_user(user_id=35e1d6073fb543fb97398c6ec63fda53)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:40.862053 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c30b650-4831-422b-979b-4dbc6efb830a admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:40.954780 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 734/1477] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:29:40 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 331 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:29:40.956218 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d6961d-1afd-4f5b-81c9-4781e48e2903 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:40.958299 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-09d6961d-1afd-4f5b-81c9-4781e48e2903 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:41.033559 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-8c30b650-4831-422b-979b-4dbc6efb830a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '35e1d6073fb543fb97398c6ec63fda53'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:41.047338 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:29:41.061919 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-8c30b650-4831-422b-979b-4dbc6efb830a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '35e1d6073fb543fb97398c6ec63fda53'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:41.062589 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-8c30b650-4831-422b-979b-4dbc6efb830a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '35e1d6073fb543fb97398c6ec63fda53'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:41.081725 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d6961d-1afd-4f5b-81c9-4781e48e2903 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:41.087424 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a197fc-14ec-4a2b-9881-f19a85df6360 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3ec6ecb85f94ef59762877d9ea33ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:41.087778 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-8c30b650-4831-422b-979b-4dbc6efb830a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '35e1d6073fb543fb97398c6ec63fda53'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:41.088782 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-8c30b650-4831-422b-979b-4dbc6efb830a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '35e1d6073fb543fb97398c6ec63fda53'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:41.091947 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a197fc-14ec-4a2b-9881-f19a85df6360 service neutron] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:41.092613 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a197fc-14ec-4a2b-9881-f19a85df6360 service neutron] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:41.093121 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a197fc-14ec-4a2b-9881-f19a85df6360 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:41.095969 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-02a197fc-14ec-4a2b-9881-f19a85df6360 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:41.123035 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 744/1478] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Nov 2 07:29:40 2022] DELETE /identity/v3/users/35e1d6073fb543fb97398c6ec63fda53 => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:29:41.123035 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c30b650-4831-422b-979b-4dbc6efb830a admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:41.123282 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-8c30b650-4831-422b-979b-4dbc6efb830a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:41.124297 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a197fc-14ec-4a2b-9881-f19a85df6360 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:41.126962 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a197fc-14ec-4a2b-9881-f19a85df6360 service neutron] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:41.147084 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 735/1479] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:29:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:29:41.148952 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c51d682-5df5-4f89-96cd-45c34e7e2be5 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:41.149480 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c51d682-5df5-4f89-96cd-45c34e7e2be5 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:41.150297 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c51d682-5df5-4f89-96cd-45c34e7e2be5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:41.205507 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-1c51d682-5df5-4f89-96cd-45c34e7e2be5 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:29:41.208094 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-1c51d682-5df5-4f89-96cd-45c34e7e2be5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:41.287056 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c30b650-4831-422b-979b-4dbc6efb830a admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:41.301913 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c1ba82-b5ad-4d5f-a190-50cde9ff5496 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43bb326fb82e4d2c83fb2eaf91860dc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:41.302589 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c1ba82-b5ad-4d5f-a190-50cde9ff5496 service cinder] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:41.305991 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c1ba82-b5ad-4d5f-a190-50cde9ff5496 service cinder] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:41.307160 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c1ba82-b5ad-4d5f-a190-50cde9ff5496 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:41.324258 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-03c1ba82-b5ad-4d5f-a190-50cde9ff5496 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:41.334012 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 736/1480] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:29:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:29:41.406019 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c51d682-5df5-4f89-96cd-45c34e7e2be5 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:41.410047 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-1c51d682-5df5-4f89-96cd-45c34e7e2be5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:41.442111 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:29:41.564832 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c1ba82-b5ad-4d5f-a190-50cde9ff5496 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:41.566344 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c1ba82-b5ad-4d5f-a190-50cde9ff5496 service cinder] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:41.592649 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 745/1481] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:29:41 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 473 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:29:41.641926 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c51d682-5df5-4f89-96cd-45c34e7e2be5 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:41.653389 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf2d80b-af34-4971-93d0-8f8574d56c02 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3ec6ecb85f94ef59762877d9ea33ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:41.656342 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf2d80b-af34-4971-93d0-8f8574d56c02 service neutron] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:41.656682 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf2d80b-af34-4971-93d0-8f8574d56c02 service neutron] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:41.656993 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf2d80b-af34-4971-93d0-8f8574d56c02 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:41.684868 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf2d80b-af34-4971-93d0-8f8574d56c02 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:41.686679 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf2d80b-af34-4971-93d0-8f8574d56c02 service neutron] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:41.717815 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 737/1482] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:29:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 07:29:41.828208 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c1ba82-b5ad-4d5f-a190-50cde9ff5496 service cinder] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:41.855006 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c1ba82-b5ad-4d5f-a190-50cde9ff5496 service cinder] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:41.859139 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf2d80b-af34-4971-93d0-8f8574d56c02 service neutron] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:41.860785 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-5cf2d80b-af34-4971-93d0-8f8574d56c02 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:41.870535 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f88fd3-634b-422e-8348-d5b3b104006e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43bb326fb82e4d2c83fb2eaf91860dc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:41.871331 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f88fd3-634b-422e-8348-d5b3b104006e service cinder] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:41.871618 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f88fd3-634b-422e-8348-d5b3b104006e service cinder] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:41.871896 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f88fd3-634b-422e-8348-d5b3b104006e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:41.874403 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-e6f88fd3-634b-422e-8348-d5b3b104006e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:42.027782 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f88fd3-634b-422e-8348-d5b3b104006e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:42.029237 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f88fd3-634b-422e-8348-d5b3b104006e service cinder] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:42.068694 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 746/1483] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:29:41 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:29:42.078734 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf2d80b-af34-4971-93d0-8f8574d56c02 service neutron] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:42.086877 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fc9d15-7ca3-4c0e-a3b2-c3ac8620f216 tempest-VolumeQuotaClassesTest-11452308 tempest-VolumeQuotaClassesTest-11452308-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359f0d29a38145419dc8d571f7f17a08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011a28d25afe452dac9be134650a1345', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:42.087756 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fc9d15-7ca3-4c0e-a3b2-c3ac8620f216 tempest-VolumeQuotaClassesTest-11452308 tempest-VolumeQuotaClassesTest-11452308-project-admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:42.088178 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fc9d15-7ca3-4c0e-a3b2-c3ac8620f216 tempest-VolumeQuotaClassesTest-11452308 tempest-VolumeQuotaClassesTest-11452308-project-admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:42.088589 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fc9d15-7ca3-4c0e-a3b2-c3ac8620f216 tempest-VolumeQuotaClassesTest-11452308 tempest-VolumeQuotaClassesTest-11452308-project-admin] PATH_INFO: `/v3/domains` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:42.090796 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fc9d15-7ca3-4c0e-a3b2-c3ac8620f216 tempest-VolumeQuotaClassesTest-11452308 tempest-VolumeQuotaClassesTest-11452308-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:29:42.091270 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fc9d15-7ca3-4c0e-a3b2-c3ac8620f216 tempest-VolumeQuotaClassesTest-11452308 tempest-VolumeQuotaClassesTest-11452308-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:42.094169 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fc9d15-7ca3-4c0e-a3b2-c3ac8620f216 tempest-VolumeQuotaClassesTest-11452308 tempest-VolumeQuotaClassesTest-11452308-project-admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:42.139522 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f88fd3-634b-422e-8348-d5b3b104006e service cinder] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:42.150168 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 738/1484] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Nov 2 07:29:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:29:42.156427 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f88fd3-634b-422e-8348-d5b3b104006e service cinder] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:42.163439 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a408da8c-6b87-4280-8758-2a29f8efe921 tempest-VolumeQuotaClassesTest-11452308 tempest-VolumeQuotaClassesTest-11452308-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359f0d29a38145419dc8d571f7f17a08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011a28d25afe452dac9be134650a1345', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:42.164064 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a408da8c-6b87-4280-8758-2a29f8efe921 tempest-VolumeQuotaClassesTest-11452308 tempest-VolumeQuotaClassesTest-11452308-project-admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:42.164357 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a408da8c-6b87-4280-8758-2a29f8efe921 tempest-VolumeQuotaClassesTest-11452308 tempest-VolumeQuotaClassesTest-11452308-project-admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:42.164620 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a408da8c-6b87-4280-8758-2a29f8efe921 tempest-VolumeQuotaClassesTest-11452308 tempest-VolumeQuotaClassesTest-11452308-project-admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:42.165453 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a408da8c-6b87-4280-8758-2a29f8efe921 tempest-VolumeQuotaClassesTest-11452308 tempest-VolumeQuotaClassesTest-11452308-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:42.168207 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a408da8c-6b87-4280-8758-2a29f8efe921 tempest-VolumeQuotaClassesTest-11452308 tempest-VolumeQuotaClassesTest-11452308-project-admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:42.187729 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:DELETE=3,keystone:SELECT=97 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:29:42.230463 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 747/1485] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Nov 2 07:29:42 2022] POST /identity/v3/projects => generated 354 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:29:42.236123 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fc9d15-7ca3-4c0e-a3b2-c3ac8620f216 tempest-VolumeQuotaClassesTest-11452308 tempest-VolumeQuotaClassesTest-11452308-project-admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:42.256658 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fc9d15-7ca3-4c0e-a3b2-c3ac8620f216 tempest-VolumeQuotaClassesTest-11452308 tempest-VolumeQuotaClassesTest-11452308-project-admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:42.307430 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0aa047-3009-4483-87f1-4008ad81df7e None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:42.308377 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0aa047-3009-4483-87f1-4008ad81df7e None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:42.308377 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0aa047-3009-4483-87f1-4008ad81df7e None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:42.310031 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 748/1486] 10.222.0.18 () {54 vars in 921 bytes} [Wed Nov 2 07:29:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:29:42.365705 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62160335-0043-47d6-af8e-b5e65d4b720b None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:42.367025 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62160335-0043-47d6-af8e-b5e65d4b720b None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:42.367439 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62160335-0043-47d6-af8e-b5e65d4b720b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:42.396266 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-62160335-0043-47d6-af8e-b5e65d4b720b None None] MFA Rules not processed for user `69e48cba3f804cf08b2477f01ae95179`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:29:42.397850 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-62160335-0043-47d6-af8e-b5e65d4b720b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:42.409072 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf126cb-52ea-4cd3-8939-e17932158132 tempest-VolumeQuotaClassesTest-11452308 tempest-VolumeQuotaClassesTest-11452308-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359f0d29a38145419dc8d571f7f17a08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011a28d25afe452dac9be134650a1345', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:42.420677 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf126cb-52ea-4cd3-8939-e17932158132 tempest-VolumeQuotaClassesTest-11452308 tempest-VolumeQuotaClassesTest-11452308-project-admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:42.421407 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf126cb-52ea-4cd3-8939-e17932158132 tempest-VolumeQuotaClassesTest-11452308 tempest-VolumeQuotaClassesTest-11452308-project-admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:42.426089 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf126cb-52ea-4cd3-8939-e17932158132 tempest-VolumeQuotaClassesTest-11452308 tempest-VolumeQuotaClassesTest-11452308-project-admin] PATH_INFO: `/v3/domains` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:42.432049 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf126cb-52ea-4cd3-8939-e17932158132 tempest-VolumeQuotaClassesTest-11452308 tempest-VolumeQuotaClassesTest-11452308-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:29:42.444109 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf126cb-52ea-4cd3-8939-e17932158132 tempest-VolumeQuotaClassesTest-11452308 tempest-VolumeQuotaClassesTest-11452308-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:42.450913 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf126cb-52ea-4cd3-8939-e17932158132 tempest-VolumeQuotaClassesTest-11452308 tempest-VolumeQuotaClassesTest-11452308-project-admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:42.505256 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 739/1487] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Nov 2 07:29:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:29:42.624980 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf126cb-52ea-4cd3-8939-e17932158132 tempest-VolumeQuotaClassesTest-11452308 tempest-VolumeQuotaClassesTest-11452308-project-admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:42.635856 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf126cb-52ea-4cd3-8939-e17932158132 tempest-VolumeQuotaClassesTest-11452308 tempest-VolumeQuotaClassesTest-11452308-project-admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:42.646802 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1f3ad7-6924-452c-a26d-4751fcddbfff tempest-VolumeQuotaClassesTest-11452308 tempest-VolumeQuotaClassesTest-11452308-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359f0d29a38145419dc8d571f7f17a08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011a28d25afe452dac9be134650a1345', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:42.646802 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1f3ad7-6924-452c-a26d-4751fcddbfff tempest-VolumeQuotaClassesTest-11452308 tempest-VolumeQuotaClassesTest-11452308-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:42.646802 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1f3ad7-6924-452c-a26d-4751fcddbfff tempest-VolumeQuotaClassesTest-11452308 tempest-VolumeQuotaClassesTest-11452308-project-admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:42.647177 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1f3ad7-6924-452c-a26d-4751fcddbfff tempest-VolumeQuotaClassesTest-11452308 tempest-VolumeQuotaClassesTest-11452308-project-admin] PATH_INFO: `/v3/projects/3bf8ce7126f44d9aa5520bfab3d98b89` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:42.650868 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1f3ad7-6924-452c-a26d-4751fcddbfff tempest-VolumeQuotaClassesTest-11452308 tempest-VolumeQuotaClassesTest-11452308-project-admin] RBAC: Authorizing `identity:delete_project(project_id=3bf8ce7126f44d9aa5520bfab3d98b89)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:42.652852 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1f3ad7-6924-452c-a26d-4751fcddbfff tempest-VolumeQuotaClassesTest-11452308 tempest-VolumeQuotaClassesTest-11452308-project-admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:42.672733 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 749/1488] 10.222.0.18 () {58 vars in 1018 bytes} [Wed Nov 2 07:29:42 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 308 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:29:42.764344 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:29:42.782391 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-4c1f3ad7-6924-452c-a26d-4751fcddbfff tempest-VolumeQuotaClassesTest-11452308 tempest-VolumeQuotaClassesTest-11452308-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3bf8ce7126f44d9aa5520bfab3d98b89', 'request_id': 'req-4c1f3ad7-6924-452c-a26d-4751fcddbfff'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:42.809381 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-4c1f3ad7-6924-452c-a26d-4751fcddbfff tempest-VolumeQuotaClassesTest-11452308 tempest-VolumeQuotaClassesTest-11452308-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3bf8ce7126f44d9aa5520bfab3d98b89', 'request_id': 'req-4c1f3ad7-6924-452c-a26d-4751fcddbfff'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:42.841346 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-4c1f3ad7-6924-452c-a26d-4751fcddbfff tempest-VolumeQuotaClassesTest-11452308 tempest-VolumeQuotaClassesTest-11452308-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3bf8ce7126f44d9aa5520bfab3d98b89', 'request_id': 'req-4c1f3ad7-6924-452c-a26d-4751fcddbfff'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:42.881264 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-4c1f3ad7-6924-452c-a26d-4751fcddbfff tempest-VolumeQuotaClassesTest-11452308 tempest-VolumeQuotaClassesTest-11452308-project-admin] The token cache is being invalidate because project 3bf8ce7126f44d9aa5520bfab3d98b89 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81501) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:29:42.881264 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-4c1f3ad7-6924-452c-a26d-4751fcddbfff tempest-VolumeQuotaClassesTest-11452308 tempest-VolumeQuotaClassesTest-11452308-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:42.884897 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-4c1f3ad7-6924-452c-a26d-4751fcddbfff tempest-VolumeQuotaClassesTest-11452308 tempest-VolumeQuotaClassesTest-11452308-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:42.902418 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 740/1489] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Nov 2 07:29:42 2022] DELETE /identity/v3/projects/3bf8ce7126f44d9aa5520bfab3d98b89 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:29:42.965756 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62160335-0043-47d6-af8e-b5e65d4b720b None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:42.969535 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-62160335-0043-47d6-af8e-b5e65d4b720b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:43.017920 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62160335-0043-47d6-af8e-b5e65d4b720b None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:43.282966 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bafc72-fa14-4e1f-8919-15521542afda service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e48cba3f804cf08b2477f01ae95179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:43.283813 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bafc72-fa14-4e1f-8919-15521542afda service nova] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:43.284271 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bafc72-fa14-4e1f-8919-15521542afda service nova] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:43.284706 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bafc72-fa14-4e1f-8919-15521542afda service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:43.298697 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-f4bafc72-fa14-4e1f-8919-15521542afda service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:43.340931 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bafc72-fa14-4e1f-8919-15521542afda service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:43.343643 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bafc72-fa14-4e1f-8919-15521542afda service nova] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:43.372272 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 750/1490] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:29:42 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:29:43.440499 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1f3ad7-6924-452c-a26d-4751fcddbfff tempest-VolumeQuotaClassesTest-11452308 tempest-VolumeQuotaClassesTest-11452308-project-admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:43.443495 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-4c1f3ad7-6924-452c-a26d-4751fcddbfff tempest-VolumeQuotaClassesTest-11452308 tempest-VolumeQuotaClassesTest-11452308-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:43.485055 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3e600d-c678-4698-b839-8b2766ca70a7 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:43.485555 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3e600d-c678-4698-b839-8b2766ca70a7 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:43.486070 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3e600d-c678-4698-b839-8b2766ca70a7 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:43.488525 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 751/1491] 10.222.0.18 () {54 vars in 921 bytes} [Wed Nov 2 07:29:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:29:43.496511 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83f87de-d065-4242-8527-8b9a28421291 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:43.497635 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83f87de-d065-4242-8527-8b9a28421291 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:43.498281 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83f87de-d065-4242-8527-8b9a28421291 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:43.552492 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-e83f87de-d065-4242-8527-8b9a28421291 None None] MFA Rules not processed for user `4f2fc6a156c54d0c9d325901d0f7f6fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:29:43.553749 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-e83f87de-d065-4242-8527-8b9a28421291 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:43.630588 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1f3ad7-6924-452c-a26d-4751fcddbfff tempest-VolumeQuotaClassesTest-11452308 tempest-VolumeQuotaClassesTest-11452308-project-admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:43.643925 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c5d18b-ddce-4bfa-9ea0-97f63f6fcf64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:43.644933 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c5d18b-ddce-4bfa-9ea0-97f63f6fcf64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:43.649318 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c5d18b-ddce-4bfa-9ea0-97f63f6fcf64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:43.649318 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c5d18b-ddce-4bfa-9ea0-97f63f6fcf64 admin admin] PATH_INFO: `/v3/users/603d63b3db5e4363b449974b99a56a44` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:43.649318 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c5d18b-ddce-4bfa-9ea0-97f63f6fcf64 admin admin] RBAC: Authorizing `identity:delete_user(user_id=603d63b3db5e4363b449974b99a56a44)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:43.654623 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c5d18b-ddce-4bfa-9ea0-97f63f6fcf64 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:43.699379 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 752/1492] 10.222.0.18 () {58 vars in 1018 bytes} [Wed Nov 2 07:29:43 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:29:43.799905 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d2118b-6f6d-47c8-9107-db7ff3df9e6a None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:43.800409 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d2118b-6f6d-47c8-9107-db7ff3df9e6a None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:43.800962 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d2118b-6f6d-47c8-9107-db7ff3df9e6a None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:43.802976 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 753/1493] 10.222.0.18 () {54 vars in 961 bytes} [Wed Nov 2 07:29:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Nov 02 07:29:43.812067 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dad7320-381a-4aa9-8bcd-902e5785c10f None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:43.812454 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dad7320-381a-4aa9-8bcd-902e5785c10f None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:43.812934 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dad7320-381a-4aa9-8bcd-902e5785c10f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:43.824683 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-b8c5d18b-ddce-4bfa-9ea0-97f63f6fcf64 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '603d63b3db5e4363b449974b99a56a44'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:43.858926 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-b8c5d18b-ddce-4bfa-9ea0-97f63f6fcf64 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '603d63b3db5e4363b449974b99a56a44'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:43.860031 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-b8c5d18b-ddce-4bfa-9ea0-97f63f6fcf64 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '603d63b3db5e4363b449974b99a56a44'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:43.865199 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:29:43.888790 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-0dad7320-381a-4aa9-8bcd-902e5785c10f None None] MFA Rules not processed for user `4f2fc6a156c54d0c9d325901d0f7f6fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:29:43.892734 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-0dad7320-381a-4aa9-8bcd-902e5785c10f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:43.893543 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-b8c5d18b-ddce-4bfa-9ea0-97f63f6fcf64 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '603d63b3db5e4363b449974b99a56a44'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:43.966274 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 754/1494] 10.222.0.18 () {58 vars in 1058 bytes} [Wed Nov 2 07:29:43 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:29:43.972303 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dad7320-381a-4aa9-8bcd-902e5785c10f None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:43.978929 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-b8c5d18b-ddce-4bfa-9ea0-97f63f6fcf64 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '603d63b3db5e4363b449974b99a56a44'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:43.995606 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dad7320-381a-4aa9-8bcd-902e5785c10f None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:44.001923 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70a093a-4dbd-446a-8ce4-c8949cc79b03 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2fc6a156c54d0c9d325901d0f7f6fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:44.003100 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70a093a-4dbd-446a-8ce4-c8949cc79b03 service placement] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:44.003100 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70a093a-4dbd-446a-8ce4-c8949cc79b03 service placement] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:44.003293 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70a093a-4dbd-446a-8ce4-c8949cc79b03 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:44.005129 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-e70a093a-4dbd-446a-8ce4-c8949cc79b03 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:44.027947 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70a093a-4dbd-446a-8ce4-c8949cc79b03 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:44.032807 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 741/1495] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Nov 2 07:29:43 2022] DELETE /identity/v3/users/603d63b3db5e4363b449974b99a56a44 => generated 0 bytes in 593 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:29:44.034083 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70a093a-4dbd-446a-8ce4-c8949cc79b03 service placement] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:44.047463 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87aa910-392f-435e-bf93-e5e1492e7749 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:44.049123 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87aa910-392f-435e-bf93-e5e1492e7749 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:44.049792 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87aa910-392f-435e-bf93-e5e1492e7749 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:44.071761 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 755/1496] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:29:43 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:29:44.150238 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70a093a-4dbd-446a-8ce4-c8949cc79b03 service placement] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:44.152173 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-e70a093a-4dbd-446a-8ce4-c8949cc79b03 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:44.169531 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-e87aa910-392f-435e-bf93-e5e1492e7749 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:29:44.170818 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-e87aa910-392f-435e-bf93-e5e1492e7749 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:44.203713 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70a093a-4dbd-446a-8ce4-c8949cc79b03 service placement] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:44.373457 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3e1dc5-fe31-4921-be29-56b59174b0e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:44.375080 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3e1dc5-fe31-4921-be29-56b59174b0e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:44.375372 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3e1dc5-fe31-4921-be29-56b59174b0e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:44.375644 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3e1dc5-fe31-4921-be29-56b59174b0e6 admin admin] PATH_INFO: `/v3/projects/afc08e9049be4671be109b98305098a7` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:44.381130 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:29:44.383429 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3e1dc5-fe31-4921-be29-56b59174b0e6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=afc08e9049be4671be109b98305098a7)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:44.396596 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3e1dc5-fe31-4921-be29-56b59174b0e6 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:44.499536 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 742/1497] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:29:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 456 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:29:44.529001 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-cc3e1dc5-fe31-4921-be29-56b59174b0e6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'afc08e9049be4671be109b98305098a7', 'request_id': 'req-cc3e1dc5-fe31-4921-be29-56b59174b0e6'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:44.545868 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-cc3e1dc5-fe31-4921-be29-56b59174b0e6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'afc08e9049be4671be109b98305098a7', 'request_id': 'req-cc3e1dc5-fe31-4921-be29-56b59174b0e6'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:44.560028 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-cc3e1dc5-fe31-4921-be29-56b59174b0e6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'afc08e9049be4671be109b98305098a7', 'request_id': 'req-cc3e1dc5-fe31-4921-be29-56b59174b0e6'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:44.562845 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87aa910-392f-435e-bf93-e5e1492e7749 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:44.564689 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-e87aa910-392f-435e-bf93-e5e1492e7749 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:44.617508 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-cc3e1dc5-fe31-4921-be29-56b59174b0e6 admin admin] The token cache is being invalidate because project afc08e9049be4671be109b98305098a7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:29:44.618135 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-cc3e1dc5-fe31-4921-be29-56b59174b0e6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:44.619358 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-cc3e1dc5-fe31-4921-be29-56b59174b0e6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:44.623301 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 756/1498] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Nov 2 07:29:44 2022] DELETE /identity/v3/projects/afc08e9049be4671be109b98305098a7 => generated 0 bytes in 473 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:29:44.627020 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3e1dc5-fe31-4921-be29-56b59174b0e6 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:44.628530 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-cc3e1dc5-fe31-4921-be29-56b59174b0e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:44.727863 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87aa910-392f-435e-bf93-e5e1492e7749 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:44.785604 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3e1dc5-fe31-4921-be29-56b59174b0e6 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:44.793583 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530af819-7918-41af-9ef1-bacc46c5c44c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:44.794755 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530af819-7918-41af-9ef1-bacc46c5c44c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:44.795272 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530af819-7918-41af-9ef1-bacc46c5c44c admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:44.795648 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530af819-7918-41af-9ef1-bacc46c5c44c admin admin] PATH_INFO: `/v3/users/ccf803f8277740dca79913ee2f8b462b` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:44.797790 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530af819-7918-41af-9ef1-bacc46c5c44c admin admin] RBAC: Authorizing `identity:delete_user(user_id=ccf803f8277740dca79913ee2f8b462b)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:44.800395 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530af819-7918-41af-9ef1-bacc46c5c44c admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:44.856832 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:29:44.901905 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01524d40-5416-424c-9374-588f3e14b386 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3ec6ecb85f94ef59762877d9ea33ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:44.902900 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01524d40-5416-424c-9374-588f3e14b386 service neutron] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:44.904142 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01524d40-5416-424c-9374-588f3e14b386 service neutron] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:44.905244 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01524d40-5416-424c-9374-588f3e14b386 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:44.909605 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-01524d40-5416-424c-9374-588f3e14b386 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:44.916978 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-530af819-7918-41af-9ef1-bacc46c5c44c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ccf803f8277740dca79913ee2f8b462b'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:44.957331 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-530af819-7918-41af-9ef1-bacc46c5c44c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ccf803f8277740dca79913ee2f8b462b'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:44.958213 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-530af819-7918-41af-9ef1-bacc46c5c44c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ccf803f8277740dca79913ee2f8b462b'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:44.965587 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01524d40-5416-424c-9374-588f3e14b386 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:44.968172 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01524d40-5416-424c-9374-588f3e14b386 service neutron] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:44.999139 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-530af819-7918-41af-9ef1-bacc46c5c44c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ccf803f8277740dca79913ee2f8b462b'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:45.000584 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-530af819-7918-41af-9ef1-bacc46c5c44c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ccf803f8277740dca79913ee2f8b462b'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:45.007108 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 743/1499] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:29:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 444 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 07:29:45.078881 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 757/1500] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Nov 2 07:29:44 2022] DELETE /identity/v3/users/ccf803f8277740dca79913ee2f8b462b => generated 0 bytes in 449 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:29:45.576281 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01524d40-5416-424c-9374-588f3e14b386 service neutron] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:45.578308 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-01524d40-5416-424c-9374-588f3e14b386 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:45.608098 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01524d40-5416-424c-9374-588f3e14b386 service neutron] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:45.707145 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4897c87f-241a-45bc-b2bb-3a5c83ba1ccc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:45.708178 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4897c87f-241a-45bc-b2bb-3a5c83ba1ccc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:45.708678 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4897c87f-241a-45bc-b2bb-3a5c83ba1ccc admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:45.709349 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4897c87f-241a-45bc-b2bb-3a5c83ba1ccc admin admin] PATH_INFO: `/v3/projects/1b705ff84b404e42a998eddc878928b0` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:45.710829 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4897c87f-241a-45bc-b2bb-3a5c83ba1ccc admin admin] RBAC: Authorizing `identity:delete_project(project_id=1b705ff84b404e42a998eddc878928b0)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:45.713892 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4897c87f-241a-45bc-b2bb-3a5c83ba1ccc admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:45.773514 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:29:45.779107 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-4897c87f-241a-45bc-b2bb-3a5c83ba1ccc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1b705ff84b404e42a998eddc878928b0', 'request_id': 'req-4897c87f-241a-45bc-b2bb-3a5c83ba1ccc'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:45.793700 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-4897c87f-241a-45bc-b2bb-3a5c83ba1ccc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1b705ff84b404e42a998eddc878928b0', 'request_id': 'req-4897c87f-241a-45bc-b2bb-3a5c83ba1ccc'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:45.811400 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-4897c87f-241a-45bc-b2bb-3a5c83ba1ccc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1b705ff84b404e42a998eddc878928b0', 'request_id': 'req-4897c87f-241a-45bc-b2bb-3a5c83ba1ccc'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:45.840454 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-4897c87f-241a-45bc-b2bb-3a5c83ba1ccc admin admin] The token cache is being invalidate because project 1b705ff84b404e42a998eddc878928b0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81501) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:29:45.840720 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-4897c87f-241a-45bc-b2bb-3a5c83ba1ccc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:45.842298 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-4897c87f-241a-45bc-b2bb-3a5c83ba1ccc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:45.843308 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 744/1501] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Nov 2 07:29:45 2022] DELETE /identity/v3/projects/1b705ff84b404e42a998eddc878928b0 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:29:45.848365 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530af819-7918-41af-9ef1-bacc46c5c44c admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:45.850607 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-530af819-7918-41af-9ef1-bacc46c5c44c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:45.871059 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530af819-7918-41af-9ef1-bacc46c5c44c admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:46.155566 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2ab95e-20e5-4d58-8d67-3718715e4c42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:46.155781 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2ab95e-20e5-4d58-8d67-3718715e4c42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:46.156016 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2ab95e-20e5-4d58-8d67-3718715e4c42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:46.156284 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2ab95e-20e5-4d58-8d67-3718715e4c42 admin admin] PATH_INFO: `/v3/users/459d8e2afac14ea780ebd0cf615a88f2` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:46.160457 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2ab95e-20e5-4d58-8d67-3718715e4c42 admin admin] RBAC: Authorizing `identity:delete_user(user_id=459d8e2afac14ea780ebd0cf615a88f2)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:46.162090 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2ab95e-20e5-4d58-8d67-3718715e4c42 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:46.346979 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-6f2ab95e-20e5-4d58-8d67-3718715e4c42 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '459d8e2afac14ea780ebd0cf615a88f2'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:46.381302 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-6f2ab95e-20e5-4d58-8d67-3718715e4c42 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '459d8e2afac14ea780ebd0cf615a88f2'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:46.382731 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-6f2ab95e-20e5-4d58-8d67-3718715e4c42 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '459d8e2afac14ea780ebd0cf615a88f2'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:46.409674 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-6f2ab95e-20e5-4d58-8d67-3718715e4c42 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '459d8e2afac14ea780ebd0cf615a88f2'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:46.411987 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-6f2ab95e-20e5-4d58-8d67-3718715e4c42 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '459d8e2afac14ea780ebd0cf615a88f2'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:46.486789 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 758/1502] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Nov 2 07:29:45 2022] DELETE /identity/v3/users/459d8e2afac14ea780ebd0cf615a88f2 => generated 0 bytes in 636 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 07:29:47.036268 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4897c87f-241a-45bc-b2bb-3a5c83ba1ccc admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:47.038495 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-4897c87f-241a-45bc-b2bb-3a5c83ba1ccc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:47.055023 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4897c87f-241a-45bc-b2bb-3a5c83ba1ccc admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:47.155027 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095385e6-4e99-444a-8502-42f04362a6c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:47.155877 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095385e6-4e99-444a-8502-42f04362a6c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:47.156246 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095385e6-4e99-444a-8502-42f04362a6c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:47.156497 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095385e6-4e99-444a-8502-42f04362a6c0 admin admin] PATH_INFO: `/v3/projects/6965bb3ff72d43c0a8dfcf873424656f` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:47.157807 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095385e6-4e99-444a-8502-42f04362a6c0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6965bb3ff72d43c0a8dfcf873424656f)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:47.160207 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095385e6-4e99-444a-8502-42f04362a6c0 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:47.236447 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-095385e6-4e99-444a-8502-42f04362a6c0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6965bb3ff72d43c0a8dfcf873424656f', 'request_id': 'req-095385e6-4e99-444a-8502-42f04362a6c0'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:47.244688 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-095385e6-4e99-444a-8502-42f04362a6c0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6965bb3ff72d43c0a8dfcf873424656f', 'request_id': 'req-095385e6-4e99-444a-8502-42f04362a6c0'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:47.264830 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-095385e6-4e99-444a-8502-42f04362a6c0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6965bb3ff72d43c0a8dfcf873424656f', 'request_id': 'req-095385e6-4e99-444a-8502-42f04362a6c0'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:47.292973 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-095385e6-4e99-444a-8502-42f04362a6c0 admin admin] The token cache is being invalidate because project 6965bb3ff72d43c0a8dfcf873424656f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81501) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:29:47.292973 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-095385e6-4e99-444a-8502-42f04362a6c0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:47.293994 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-095385e6-4e99-444a-8502-42f04362a6c0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:47.295965 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 745/1503] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Nov 2 07:29:47 2022] DELETE /identity/v3/projects/6965bb3ff72d43c0a8dfcf873424656f => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:29:47.301506 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7e204f-ddfb-4b14-8ee8-d971310641d5 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:47.302073 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7e204f-ddfb-4b14-8ee8-d971310641d5 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:47.302073 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7e204f-ddfb-4b14-8ee8-d971310641d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:47.341397 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095385e6-4e99-444a-8502-42f04362a6c0 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:47.342982 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-095385e6-4e99-444a-8502-42f04362a6c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:47.343659 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-6f7e204f-ddfb-4b14-8ee8-d971310641d5 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:29:47.345928 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-6f7e204f-ddfb-4b14-8ee8-d971310641d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:47.374153 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:DELETE=18,keystone:SELECT=80,keystone:INSERT=2 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:29:47.480027 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095385e6-4e99-444a-8502-42f04362a6c0 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:47.490491 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 759/1504] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:29:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:29:47.498568 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1b061d-2627-42c9-be5a-d8b8dc19834b None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:47.498568 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1b061d-2627-42c9-be5a-d8b8dc19834b None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:47.498568 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1b061d-2627-42c9-be5a-d8b8dc19834b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:47.501514 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4528375-bac8-4b88-a3b9-0dc1beb28701 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:47.502520 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4528375-bac8-4b88-a3b9-0dc1beb28701 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:47.502937 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4528375-bac8-4b88-a3b9-0dc1beb28701 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:47.503425 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4528375-bac8-4b88-a3b9-0dc1beb28701 admin admin] PATH_INFO: `/v3/projects/d1eab68550f348c78e18ba704c313f82` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:47.505102 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4528375-bac8-4b88-a3b9-0dc1beb28701 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d1eab68550f348c78e18ba704c313f82)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:47.507976 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4528375-bac8-4b88-a3b9-0dc1beb28701 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:47.511627 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:29:47.522781 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-cf1b061d-2627-42c9-be5a-d8b8dc19834b None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:29:47.524209 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-cf1b061d-2627-42c9-be5a-d8b8dc19834b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:47.536972 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 760/1505] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:29:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:29:47.538480 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1b061d-2627-42c9-be5a-d8b8dc19834b None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:47.551637 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1b061d-2627-42c9-be5a-d8b8dc19834b None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:47.565319 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-f4528375-bac8-4b88-a3b9-0dc1beb28701 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd1eab68550f348c78e18ba704c313f82', 'request_id': 'req-f4528375-bac8-4b88-a3b9-0dc1beb28701'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:47.572484 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-f4528375-bac8-4b88-a3b9-0dc1beb28701 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd1eab68550f348c78e18ba704c313f82', 'request_id': 'req-f4528375-bac8-4b88-a3b9-0dc1beb28701'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:47.581046 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-f4528375-bac8-4b88-a3b9-0dc1beb28701 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd1eab68550f348c78e18ba704c313f82', 'request_id': 'req-f4528375-bac8-4b88-a3b9-0dc1beb28701'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:47.593738 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-f4528375-bac8-4b88-a3b9-0dc1beb28701 admin admin] The token cache is being invalidate because project d1eab68550f348c78e18ba704c313f82 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81501) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:29:47.594033 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-f4528375-bac8-4b88-a3b9-0dc1beb28701 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:47.594827 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-f4528375-bac8-4b88-a3b9-0dc1beb28701 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:47.599837 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 746/1506] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Nov 2 07:29:47 2022] DELETE /identity/v3/projects/d1eab68550f348c78e18ba704c313f82 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:29:47.601221 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaa8cbf-fbb1-49d5-bddd-7be2faa0bf7c None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:47.603126 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaa8cbf-fbb1-49d5-bddd-7be2faa0bf7c None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:47.603434 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaa8cbf-fbb1-49d5-bddd-7be2faa0bf7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:47.630735 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877aeb59-ef04-4e8c-adf5-cc63f7fc2191 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:47.631422 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877aeb59-ef04-4e8c-adf5-cc63f7fc2191 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:47.631706 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877aeb59-ef04-4e8c-adf5-cc63f7fc2191 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:47.631976 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877aeb59-ef04-4e8c-adf5-cc63f7fc2191 admin admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:47.632632 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877aeb59-ef04-4e8c-adf5-cc63f7fc2191 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:29:47.632935 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877aeb59-ef04-4e8c-adf5-cc63f7fc2191 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:47.634704 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877aeb59-ef04-4e8c-adf5-cc63f7fc2191 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:47.637329 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-9aaa8cbf-fbb1-49d5-bddd-7be2faa0bf7c None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:29:47.642640 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-9aaa8cbf-fbb1-49d5-bddd-7be2faa0bf7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:47.660162 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 747/1507] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:29:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:29:47.666528 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 761/1508] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Nov 2 07:29:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:29:47.671324 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aaa8cbf-fbb1-49d5-bddd-7be2faa0bf7c None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:47.671892 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a156cd7b-f652-4756-8d04-161c25b684f3 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:47.672174 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a156cd7b-f652-4756-8d04-161c25b684f3 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:47.672513 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a156cd7b-f652-4756-8d04-161c25b684f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:47.673344 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-9aaa8cbf-fbb1-49d5-bddd-7be2faa0bf7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:47.689337 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aaa8cbf-fbb1-49d5-bddd-7be2faa0bf7c None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:47.694015 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-a156cd7b-f652-4756-8d04-161c25b684f3 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:29:47.694873 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7933328-3852-4cde-9ba9-17892f47318d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:47.695294 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-a156cd7b-f652-4756-8d04-161c25b684f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:47.695429 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7933328-3852-4cde-9ba9-17892f47318d admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:47.695619 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7933328-3852-4cde-9ba9-17892f47318d admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:47.695895 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7933328-3852-4cde-9ba9-17892f47318d admin admin] PATH_INFO: `/v3/projects` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:47.696661 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7933328-3852-4cde-9ba9-17892f47318d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:47.699047 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7933328-3852-4cde-9ba9-17892f47318d admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:47.721443 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 762/1509] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:29:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:29:47.728062 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a156cd7b-f652-4756-8d04-161c25b684f3 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:47.736786 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a156cd7b-f652-4756-8d04-161c25b684f3 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:47.739907 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 748/1510] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Nov 2 07:29:47 2022] POST /identity/v3/projects => generated 357 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 07:29:47.744173 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7933328-3852-4cde-9ba9-17892f47318d admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:47.746464 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567f18d6-29d4-4492-89bf-15bb73aa25f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:47.747210 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567f18d6-29d4-4492-89bf-15bb73aa25f2 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:47.747451 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567f18d6-29d4-4492-89bf-15bb73aa25f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:47.747755 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567f18d6-29d4-4492-89bf-15bb73aa25f2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:47.748597 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567f18d6-29d4-4492-89bf-15bb73aa25f2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:29:47.748804 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567f18d6-29d4-4492-89bf-15bb73aa25f2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:47.750659 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567f18d6-29d4-4492-89bf-15bb73aa25f2 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:47.770804 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 763/1511] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Nov 2 07:29:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:29:47.775363 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567f18d6-29d4-4492-89bf-15bb73aa25f2 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:47.788296 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567f18d6-29d4-4492-89bf-15bb73aa25f2 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:47.842391 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7933328-3852-4cde-9ba9-17892f47318d admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:47.850910 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abeb9c7a-4b3f-4e51-882b-3ee5ca35f73e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:47.856605 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abeb9c7a-4b3f-4e51-882b-3ee5ca35f73e admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:47.856924 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abeb9c7a-4b3f-4e51-882b-3ee5ca35f73e admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:47.857784 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abeb9c7a-4b3f-4e51-882b-3ee5ca35f73e admin admin] PATH_INFO: `/v3/users` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:47.858207 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abeb9c7a-4b3f-4e51-882b-3ee5ca35f73e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:47.860782 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abeb9c7a-4b3f-4e51-882b-3ee5ca35f73e admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:47.861952 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7429b5d-836e-46a3-b6bc-87df79a66da6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:47.862595 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7429b5d-836e-46a3-b6bc-87df79a66da6 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:47.862881 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7429b5d-836e-46a3-b6bc-87df79a66da6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:47.863251 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7429b5d-836e-46a3-b6bc-87df79a66da6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:47.864155 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7429b5d-836e-46a3-b6bc-87df79a66da6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:47.866030 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7429b5d-836e-46a3-b6bc-87df79a66da6 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:47.895425 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 749/1512] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Nov 2 07:29:47 2022] POST /identity/v3/users => generated 279 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:29:47.896294 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abeb9c7a-4b3f-4e51-882b-3ee5ca35f73e admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:47.900574 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-abeb9c7a-4b3f-4e51-882b-3ee5ca35f73e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:47.908301 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 764/1513] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Nov 2 07:29:47 2022] POST /identity/v3/projects => generated 409 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:29:47.909259 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7429b5d-836e-46a3-b6bc-87df79a66da6 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:47.915536 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abeb9c7a-4b3f-4e51-882b-3ee5ca35f73e admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:47.916064 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-f7429b5d-836e-46a3-b6bc-87df79a66da6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:47.932478 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:29:48.015124 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:29:48.017458 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7429b5d-836e-46a3-b6bc-87df79a66da6 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:48.026332 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77daadb-39e3-48c8-8026-bdf78a0fe765 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:48.027501 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77daadb-39e3-48c8-8026-bdf78a0fe765 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:48.030674 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77daadb-39e3-48c8-8026-bdf78a0fe765 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:48.031613 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77daadb-39e3-48c8-8026-bdf78a0fe765 admin admin] PATH_INFO: `/v3/projects/5f7250b8cb574e96a41dd8123062dc3f` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:48.034055 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77daadb-39e3-48c8-8026-bdf78a0fe765 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5f7250b8cb574e96a41dd8123062dc3f)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:48.034652 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee99038-a413-4549-9dc6-1888f2a9344c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:48.036281 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee99038-a413-4549-9dc6-1888f2a9344c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:48.037037 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77daadb-39e3-48c8-8026-bdf78a0fe765 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:48.037350 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee99038-a413-4549-9dc6-1888f2a9344c admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:48.038522 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee99038-a413-4549-9dc6-1888f2a9344c admin admin] PATH_INFO: `/v3/users/cf819892970e44cbad87ddf5f54aebfc` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:48.039916 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee99038-a413-4549-9dc6-1888f2a9344c admin admin] RBAC: Authorizing `identity:delete_user(user_id=cf819892970e44cbad87ddf5f54aebfc)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:48.043790 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee99038-a413-4549-9dc6-1888f2a9344c admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:48.117135 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-d77daadb-39e3-48c8-8026-bdf78a0fe765 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5f7250b8cb574e96a41dd8123062dc3f', 'request_id': 'req-d77daadb-39e3-48c8-8026-bdf78a0fe765'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:48.130093 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-d77daadb-39e3-48c8-8026-bdf78a0fe765 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5f7250b8cb574e96a41dd8123062dc3f', 'request_id': 'req-d77daadb-39e3-48c8-8026-bdf78a0fe765'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:48.137367 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-d77daadb-39e3-48c8-8026-bdf78a0fe765 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5f7250b8cb574e96a41dd8123062dc3f', 'request_id': 'req-d77daadb-39e3-48c8-8026-bdf78a0fe765'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:48.145968 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-6ee99038-a413-4549-9dc6-1888f2a9344c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cf819892970e44cbad87ddf5f54aebfc'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:48.156351 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-6ee99038-a413-4549-9dc6-1888f2a9344c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cf819892970e44cbad87ddf5f54aebfc'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:48.157300 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-6ee99038-a413-4549-9dc6-1888f2a9344c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cf819892970e44cbad87ddf5f54aebfc'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:48.160462 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-d77daadb-39e3-48c8-8026-bdf78a0fe765 admin admin] The token cache is being invalidate because project 5f7250b8cb574e96a41dd8123062dc3f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:29:48.160764 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-d77daadb-39e3-48c8-8026-bdf78a0fe765 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:48.161819 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-d77daadb-39e3-48c8-8026-bdf78a0fe765 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:48.163412 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 765/1514] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Nov 2 07:29:47 2022] DELETE /identity/v3/projects/5f7250b8cb574e96a41dd8123062dc3f => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:29:48.164319 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77daadb-39e3-48c8-8026-bdf78a0fe765 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:48.165746 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-d77daadb-39e3-48c8-8026-bdf78a0fe765 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:48.166494 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-6ee99038-a413-4549-9dc6-1888f2a9344c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cf819892970e44cbad87ddf5f54aebfc'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:48.181926 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-6ee99038-a413-4549-9dc6-1888f2a9344c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cf819892970e44cbad87ddf5f54aebfc'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:48.191857 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 750/1515] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Nov 2 07:29:47 2022] DELETE /identity/v3/users/cf819892970e44cbad87ddf5f54aebfc => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:29:48.191857 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee99038-a413-4549-9dc6-1888f2a9344c admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:48.191857 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-6ee99038-a413-4549-9dc6-1888f2a9344c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:48.261028 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77daadb-39e3-48c8-8026-bdf78a0fe765 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:48.296891 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee99038-a413-4549-9dc6-1888f2a9344c admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:48.302116 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a8b20e-b98c-4c1b-a02a-9b39b0c9a8da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:48.302993 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a8b20e-b98c-4c1b-a02a-9b39b0c9a8da admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:48.303743 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a8b20e-b98c-4c1b-a02a-9b39b0c9a8da admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:48.304166 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a8b20e-b98c-4c1b-a02a-9b39b0c9a8da admin admin] PATH_INFO: `/v3/users` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:48.304949 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a8b20e-b98c-4c1b-a02a-9b39b0c9a8da admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:48.307138 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a8b20e-b98c-4c1b-a02a-9b39b0c9a8da admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:48.337634 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 751/1516] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Nov 2 07:29:48 2022] POST /identity/v3/users => generated 305 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:29:48.343757 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a8b20e-b98c-4c1b-a02a-9b39b0c9a8da admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:48.349086 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-33a8b20e-b98c-4c1b-a02a-9b39b0c9a8da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:48.359964 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:29:48.368987 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a8b20e-b98c-4c1b-a02a-9b39b0c9a8da admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:48.375422 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1693f7a8-fcf9-4d63-9685-9a90480320df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:48.376246 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1693f7a8-fcf9-4d63-9685-9a90480320df admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:48.376654 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1693f7a8-fcf9-4d63-9685-9a90480320df admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:48.376821 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1693f7a8-fcf9-4d63-9685-9a90480320df admin admin] PATH_INFO: `/v3/users/359f0d29a38145419dc8d571f7f17a08` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:48.378001 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1693f7a8-fcf9-4d63-9685-9a90480320df admin admin] RBAC: Authorizing `identity:delete_user(user_id=359f0d29a38145419dc8d571f7f17a08)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:48.380179 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1693f7a8-fcf9-4d63-9685-9a90480320df admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:48.409201 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87163433-bbc5-4e1d-8494-ee43561f0db3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:48.409538 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87163433-bbc5-4e1d-8494-ee43561f0db3 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:48.409737 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87163433-bbc5-4e1d-8494-ee43561f0db3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:48.409990 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87163433-bbc5-4e1d-8494-ee43561f0db3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:48.410986 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87163433-bbc5-4e1d-8494-ee43561f0db3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:48.412579 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87163433-bbc5-4e1d-8494-ee43561f0db3 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:48.427140 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 766/1517] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:29:48 2022] GET /identity/v3/roles => generated 1420 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:29:48.428893 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87163433-bbc5-4e1d-8494-ee43561f0db3 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:48.432430 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-87163433-bbc5-4e1d-8494-ee43561f0db3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:48.450049 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87163433-bbc5-4e1d-8494-ee43561f0db3 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:48.456668 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6fa85b-02af-4a51-9b4f-aac71f4b375a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:48.457948 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6fa85b-02af-4a51-9b4f-aac71f4b375a admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:48.457948 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6fa85b-02af-4a51-9b4f-aac71f4b375a admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:48.458215 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6fa85b-02af-4a51-9b4f-aac71f4b375a admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:48.459026 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6fa85b-02af-4a51-9b4f-aac71f4b375a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:48.461470 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6fa85b-02af-4a51-9b4f-aac71f4b375a admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:48.469450 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-1693f7a8-fcf9-4d63-9685-9a90480320df admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '359f0d29a38145419dc8d571f7f17a08'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:48.474016 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 767/1518] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:29:48 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:29:48.486753 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6fa85b-02af-4a51-9b4f-aac71f4b375a admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:48.490600 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-1693f7a8-fcf9-4d63-9685-9a90480320df admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '359f0d29a38145419dc8d571f7f17a08'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:48.493664 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-1693f7a8-fcf9-4d63-9685-9a90480320df admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '359f0d29a38145419dc8d571f7f17a08'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:48.498275 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=77,keystone:DELETE=18,keystone:INSERT=5 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:29:48.509935 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-1693f7a8-fcf9-4d63-9685-9a90480320df admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '359f0d29a38145419dc8d571f7f17a08'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:48.513331 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6fa85b-02af-4a51-9b4f-aac71f4b375a admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:48.519469 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d8861f-79c2-4c33-b2bb-562f976920cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:48.520092 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d8861f-79c2-4c33-b2bb-562f976920cb admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:48.520373 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d8861f-79c2-4c33-b2bb-562f976920cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:48.520645 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d8861f-79c2-4c33-b2bb-562f976920cb admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:48.521461 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d8861f-79c2-4c33-b2bb-562f976920cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:48.526107 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d8861f-79c2-4c33-b2bb-562f976920cb admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:48.546391 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 768/1519] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:29:48 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:29:48.548087 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d8861f-79c2-4c33-b2bb-562f976920cb admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:48.554080 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-93d8861f-79c2-4c33-b2bb-562f976920cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:48.577415 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d8861f-79c2-4c33-b2bb-562f976920cb admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:48.578558 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-1693f7a8-fcf9-4d63-9685-9a90480320df admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '359f0d29a38145419dc8d571f7f17a08'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:48.583570 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05898627-e249-4c5f-80bb-d19095235093 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:48.584966 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05898627-e249-4c5f-80bb-d19095235093 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:48.585254 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05898627-e249-4c5f-80bb-d19095235093 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:48.585537 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05898627-e249-4c5f-80bb-d19095235093 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:48.586220 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05898627-e249-4c5f-80bb-d19095235093 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:48.588322 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05898627-e249-4c5f-80bb-d19095235093 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:48.590828 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 752/1520] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Nov 2 07:29:48 2022] DELETE /identity/v3/users/359f0d29a38145419dc8d571f7f17a08 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:29:48.591934 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1693f7a8-fcf9-4d63-9685-9a90480320df admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:48.596378 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-1693f7a8-fcf9-4d63-9685-9a90480320df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:48.605307 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 769/1521] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:29:48 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:29:48.606446 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05898627-e249-4c5f-80bb-d19095235093 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:48.608858 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-05898627-e249-4c5f-80bb-d19095235093 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:48.626095 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05898627-e249-4c5f-80bb-d19095235093 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:48.733329 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1693f7a8-fcf9-4d63-9685-9a90480320df admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:48.752424 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0626edd-3a91-4eec-8dd4-898d20911cc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:48.755223 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0626edd-3a91-4eec-8dd4-898d20911cc6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:48.757873 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0626edd-3a91-4eec-8dd4-898d20911cc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:48.758185 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0626edd-3a91-4eec-8dd4-898d20911cc6 admin admin] PATH_INFO: `/v3/projects/1743209f240a41f989db69285faba483` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:48.766386 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0626edd-3a91-4eec-8dd4-898d20911cc6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1743209f240a41f989db69285faba483)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:48.768950 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0626edd-3a91-4eec-8dd4-898d20911cc6 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:48.780697 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176874cb-1667-4ac2-b44d-6b16c1704292 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:48.782444 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176874cb-1667-4ac2-b44d-6b16c1704292 admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:48.782870 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176874cb-1667-4ac2-b44d-6b16c1704292 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:48.783285 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176874cb-1667-4ac2-b44d-6b16c1704292 admin admin] PATH_INFO: `/v3/projects/91af8acabef14e55b88cddae6628cada/users/5605da3621a446c598227d84d80cc45c/roles/b494fedd00f24f0b84a529e8672a156a` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:48.802927 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176874cb-1667-4ac2-b44d-6b16c1704292 admin admin] RBAC: Authorizing `identity:create_grant(project_id=91af8acabef14e55b88cddae6628cada, user_id=5605da3621a446c598227d84d80cc45c, role_id=b494fedd00f24f0b84a529e8672a156a)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:48.805875 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176874cb-1667-4ac2-b44d-6b16c1704292 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:48.834960 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 770/1522] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Nov 2 07:29:48 2022] PUT /identity/v3/projects/91af8acabef14e55b88cddae6628cada/users/5605da3621a446c598227d84d80cc45c/roles/b494fedd00f24f0b84a529e8672a156a => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:29:48.834960 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176874cb-1667-4ac2-b44d-6b16c1704292 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:48.874008 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-b0626edd-3a91-4eec-8dd4-898d20911cc6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1743209f240a41f989db69285faba483', 'request_id': 'req-b0626edd-3a91-4eec-8dd4-898d20911cc6'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:48.885944 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-b0626edd-3a91-4eec-8dd4-898d20911cc6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1743209f240a41f989db69285faba483', 'request_id': 'req-b0626edd-3a91-4eec-8dd4-898d20911cc6'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:48.908545 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-b0626edd-3a91-4eec-8dd4-898d20911cc6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1743209f240a41f989db69285faba483', 'request_id': 'req-b0626edd-3a91-4eec-8dd4-898d20911cc6'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:48.925159 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-b0626edd-3a91-4eec-8dd4-898d20911cc6 admin admin] The token cache is being invalidate because project 1743209f240a41f989db69285faba483 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81501) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:29:48.925460 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-b0626edd-3a91-4eec-8dd4-898d20911cc6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:48.926755 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-b0626edd-3a91-4eec-8dd4-898d20911cc6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:48.928275 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 753/1523] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Nov 2 07:29:48 2022] DELETE /identity/v3/projects/1743209f240a41f989db69285faba483 => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:29:48.931651 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3111571d-7998-46f5-891b-ea72e2f60f19 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:48.931962 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3111571d-7998-46f5-891b-ea72e2f60f19 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:48.932234 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3111571d-7998-46f5-891b-ea72e2f60f19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:48.941590 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176874cb-1667-4ac2-b44d-6b16c1704292 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:48.984836 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-3111571d-7998-46f5-891b-ea72e2f60f19 None None] MFA Rules not processed for user `5605da3621a446c598227d84d80cc45c`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:29:48.987049 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-3111571d-7998-46f5-891b-ea72e2f60f19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:49.009576 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:29:49.047541 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa5c711-ed56-4e3a-a47c-69d3a6a846d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:49.048331 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa5c711-ed56-4e3a-a47c-69d3a6a846d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:49.048994 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa5c711-ed56-4e3a-a47c-69d3a6a846d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:49.049364 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa5c711-ed56-4e3a-a47c-69d3a6a846d4 admin admin] PATH_INFO: `/v3/projects/7f0493557f3543cd9cc6986fd46a9fe0/users/6a49ee6dd9ac42e6b08f3bbfc85310b4/roles/b494fedd00f24f0b84a529e8672a156a` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:49.068838 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:29:49.078294 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa5c711-ed56-4e3a-a47c-69d3a6a846d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f0493557f3543cd9cc6986fd46a9fe0, user_id=6a49ee6dd9ac42e6b08f3bbfc85310b4, role_id=b494fedd00f24f0b84a529e8672a156a)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:49.081173 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa5c711-ed56-4e3a-a47c-69d3a6a846d4 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:49.086541 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 754/1524] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:29:48 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:29:49.087825 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3111571d-7998-46f5-891b-ea72e2f60f19 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:49.090349 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-3111571d-7998-46f5-891b-ea72e2f60f19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:49.104762 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3111571d-7998-46f5-891b-ea72e2f60f19 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:49.114509 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 771/1525] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Nov 2 07:29:48 2022] PUT /identity/v3/projects/7f0493557f3543cd9cc6986fd46a9fe0/users/6a49ee6dd9ac42e6b08f3bbfc85310b4/roles/b494fedd00f24f0b84a529e8672a156a => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:29:49.116257 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d10d05-cf90-4463-b685-0b6784880c7b None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:49.116547 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d10d05-cf90-4463-b685-0b6784880c7b None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:49.117082 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d10d05-cf90-4463-b685-0b6784880c7b None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:49.121007 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 772/1526] 10.222.0.18 () {54 vars in 921 bytes} [Wed Nov 2 07:29:49 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:29:49.122183 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d10d05-cf90-4463-b685-0b6784880c7b None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:49.124667 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-a2d10d05-cf90-4463-b685-0b6784880c7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:49.185949 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2113ab8-94e9-4ef7-a30f-7dab10943cf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:49.186656 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2113ab8-94e9-4ef7-a30f-7dab10943cf2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:49.186937 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2113ab8-94e9-4ef7-a30f-7dab10943cf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:49.187487 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2113ab8-94e9-4ef7-a30f-7dab10943cf2 admin admin] PATH_INFO: `/v3/users/709bda2cecbb40318182b0db9d7a0901` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:49.188764 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2113ab8-94e9-4ef7-a30f-7dab10943cf2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=709bda2cecbb40318182b0db9d7a0901)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:49.190858 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2113ab8-94e9-4ef7-a30f-7dab10943cf2 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:49.227155 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d10d05-cf90-4463-b685-0b6784880c7b None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:49.233339 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5090e5db-e54b-4ed0-9eef-c7514b6352f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:49.233966 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5090e5db-e54b-4ed0-9eef-c7514b6352f9 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:49.234275 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5090e5db-e54b-4ed0-9eef-c7514b6352f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:49.234580 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5090e5db-e54b-4ed0-9eef-c7514b6352f9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:49.235889 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5090e5db-e54b-4ed0-9eef-c7514b6352f9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:49.237957 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5090e5db-e54b-4ed0-9eef-c7514b6352f9 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:49.256125 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-f2113ab8-94e9-4ef7-a30f-7dab10943cf2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '709bda2cecbb40318182b0db9d7a0901'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:49.266386 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-f2113ab8-94e9-4ef7-a30f-7dab10943cf2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '709bda2cecbb40318182b0db9d7a0901'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:49.267026 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-f2113ab8-94e9-4ef7-a30f-7dab10943cf2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '709bda2cecbb40318182b0db9d7a0901'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:49.270468 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 773/1527] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Nov 2 07:29:49 2022] POST /identity/v3/projects => generated 355 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:29:49.272363 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17aaed0c-4698-498e-889f-b674a2a2d59c None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:49.272363 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17aaed0c-4698-498e-889f-b674a2a2d59c None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:49.272629 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17aaed0c-4698-498e-889f-b674a2a2d59c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:49.279839 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-f2113ab8-94e9-4ef7-a30f-7dab10943cf2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '709bda2cecbb40318182b0db9d7a0901'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:49.296788 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-f2113ab8-94e9-4ef7-a30f-7dab10943cf2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '709bda2cecbb40318182b0db9d7a0901'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:49.305788 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 755/1528] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Nov 2 07:29:49 2022] DELETE /identity/v3/users/709bda2cecbb40318182b0db9d7a0901 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:29:49.306265 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-17aaed0c-4698-498e-889f-b674a2a2d59c None None] MFA Rules not processed for user `6a49ee6dd9ac42e6b08f3bbfc85310b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:29:49.307257 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3ef8a3-1e50-4f62-9d5b-917c91836191 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:49.307542 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3ef8a3-1e50-4f62-9d5b-917c91836191 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:49.307903 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3ef8a3-1e50-4f62-9d5b-917c91836191 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:49.308082 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-17aaed0c-4698-498e-889f-b674a2a2d59c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:49.330977 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-be3ef8a3-1e50-4f62-9d5b-917c91836191 None None] MFA Rules not processed for user `69e48cba3f804cf08b2477f01ae95179`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:29:49.332171 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-be3ef8a3-1e50-4f62-9d5b-917c91836191 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:49.399967 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 774/1529] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:29:49 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:29:49.400974 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17aaed0c-4698-498e-889f-b674a2a2d59c None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:49.404065 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-17aaed0c-4698-498e-889f-b674a2a2d59c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:49.427268 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 756/1530] 10.222.0.18 () {58 vars in 1018 bytes} [Wed Nov 2 07:29:49 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:29:49.428255 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3ef8a3-1e50-4f62-9d5b-917c91836191 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:49.429874 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-be3ef8a3-1e50-4f62-9d5b-917c91836191 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:49.444821 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:29:49.455023 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:29:49.488853 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17aaed0c-4698-498e-889f-b674a2a2d59c None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:49.494945 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5a12cd-d302-4158-8454-6be74e6adad8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:49.496546 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5a12cd-d302-4158-8454-6be74e6adad8 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:49.498391 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5a12cd-d302-4158-8454-6be74e6adad8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:49.498391 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5a12cd-d302-4158-8454-6be74e6adad8 admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:49.498575 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5a12cd-d302-4158-8454-6be74e6adad8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:49.500573 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5a12cd-d302-4158-8454-6be74e6adad8 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:49.517195 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3ef8a3-1e50-4f62-9d5b-917c91836191 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:49.528362 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd754e03-6749-4836-ad0f-b5fa5297b72d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:49.529064 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd754e03-6749-4836-ad0f-b5fa5297b72d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:49.529346 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd754e03-6749-4836-ad0f-b5fa5297b72d admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:49.529620 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd754e03-6749-4836-ad0f-b5fa5297b72d admin admin] PATH_INFO: `/v3/projects/011a28d25afe452dac9be134650a1345` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:49.530754 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 775/1531] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Nov 2 07:29:49 2022] POST /identity/v3/users => generated 284 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:29:49.531036 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd754e03-6749-4836-ad0f-b5fa5297b72d admin admin] RBAC: Authorizing `identity:delete_project(project_id=011a28d25afe452dac9be134650a1345)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:49.531610 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5a12cd-d302-4158-8454-6be74e6adad8 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:49.532833 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd754e03-6749-4836-ad0f-b5fa5297b72d admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:49.533743 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-ed5a12cd-d302-4158-8454-6be74e6adad8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:49.546998 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5a12cd-d302-4158-8454-6be74e6adad8 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:49.551836 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae98e98-1e19-425c-b2f9-9ee69f21be71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:49.552540 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae98e98-1e19-425c-b2f9-9ee69f21be71 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:49.552751 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae98e98-1e19-425c-b2f9-9ee69f21be71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:49.552990 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae98e98-1e19-425c-b2f9-9ee69f21be71 admin admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:49.553766 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae98e98-1e19-425c-b2f9-9ee69f21be71 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:49.555707 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae98e98-1e19-425c-b2f9-9ee69f21be71 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:49.583286 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 776/1532] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Nov 2 07:29:49 2022] POST /identity/v3/projects => generated 409 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:29:49.584214 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae98e98-1e19-425c-b2f9-9ee69f21be71 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:49.586636 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-3ae98e98-1e19-425c-b2f9-9ee69f21be71 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:49.586777 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-fd754e03-6749-4836-ad0f-b5fa5297b72d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '011a28d25afe452dac9be134650a1345', 'request_id': 'req-fd754e03-6749-4836-ad0f-b5fa5297b72d'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:49.595197 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-fd754e03-6749-4836-ad0f-b5fa5297b72d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '011a28d25afe452dac9be134650a1345', 'request_id': 'req-fd754e03-6749-4836-ad0f-b5fa5297b72d'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:49.604460 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-fd754e03-6749-4836-ad0f-b5fa5297b72d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '011a28d25afe452dac9be134650a1345', 'request_id': 'req-fd754e03-6749-4836-ad0f-b5fa5297b72d'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:49.617687 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-fd754e03-6749-4836-ad0f-b5fa5297b72d admin admin] The token cache is being invalidate because project 011a28d25afe452dac9be134650a1345 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81501) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:29:49.617975 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-fd754e03-6749-4836-ad0f-b5fa5297b72d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:49.618657 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-fd754e03-6749-4836-ad0f-b5fa5297b72d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:49.620501 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 757/1533] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Nov 2 07:29:49 2022] DELETE /identity/v3/projects/011a28d25afe452dac9be134650a1345 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:29:49.621355 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd754e03-6749-4836-ad0f-b5fa5297b72d admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:49.622705 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-fd754e03-6749-4836-ad0f-b5fa5297b72d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:49.692851 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae98e98-1e19-425c-b2f9-9ee69f21be71 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:49.705804 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5155f40b-05c1-4f82-b6f3-7de2d893470a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e48cba3f804cf08b2477f01ae95179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:49.706765 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5155f40b-05c1-4f82-b6f3-7de2d893470a service nova] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:49.706999 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd754e03-6749-4836-ad0f-b5fa5297b72d admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:49.707422 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5155f40b-05c1-4f82-b6f3-7de2d893470a service nova] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:49.707903 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5155f40b-05c1-4f82-b6f3-7de2d893470a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:49.710073 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-5155f40b-05c1-4f82-b6f3-7de2d893470a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:49.733424 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5155f40b-05c1-4f82-b6f3-7de2d893470a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:49.740373 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5155f40b-05c1-4f82-b6f3-7de2d893470a service nova] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:49.753333 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 777/1534] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:29:49 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:29:49.754109 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5155f40b-05c1-4f82-b6f3-7de2d893470a service nova] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:49.755537 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-5155f40b-05c1-4f82-b6f3-7de2d893470a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:49.771597 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:29:49.815900 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f5bedf-2223-4430-ae9d-f08643164822 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:49.816602 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f5bedf-2223-4430-ae9d-f08643164822 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:49.816894 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f5bedf-2223-4430-ae9d-f08643164822 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:49.817167 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f5bedf-2223-4430-ae9d-f08643164822 admin admin] PATH_INFO: `/v3/projects/718f8ac6458a47f18440c145482ca9f8` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:49.821910 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f5bedf-2223-4430-ae9d-f08643164822 admin admin] RBAC: Authorizing `identity:delete_project(project_id=718f8ac6458a47f18440c145482ca9f8)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:49.834168 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f5bedf-2223-4430-ae9d-f08643164822 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:49.897681 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-99f5bedf-2223-4430-ae9d-f08643164822 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '718f8ac6458a47f18440c145482ca9f8', 'request_id': 'req-99f5bedf-2223-4430-ae9d-f08643164822'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:49.898634 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5155f40b-05c1-4f82-b6f3-7de2d893470a service nova] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:49.907510 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-99f5bedf-2223-4430-ae9d-f08643164822 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '718f8ac6458a47f18440c145482ca9f8', 'request_id': 'req-99f5bedf-2223-4430-ae9d-f08643164822'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:49.926014 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-99f5bedf-2223-4430-ae9d-f08643164822 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '718f8ac6458a47f18440c145482ca9f8', 'request_id': 'req-99f5bedf-2223-4430-ae9d-f08643164822'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:49.949392 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-99f5bedf-2223-4430-ae9d-f08643164822 admin admin] The token cache is being invalidate because project 718f8ac6458a47f18440c145482ca9f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81501) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:29:49.949810 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-99f5bedf-2223-4430-ae9d-f08643164822 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:49.950671 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-99f5bedf-2223-4430-ae9d-f08643164822 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:49.954053 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 758/1535] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Nov 2 07:29:49 2022] DELETE /identity/v3/projects/718f8ac6458a47f18440c145482ca9f8 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:29:49.955117 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f5bedf-2223-4430-ae9d-f08643164822 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:49.956751 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-99f5bedf-2223-4430-ae9d-f08643164822 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:49.968808 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:29:50.022699 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bb26cc-1b84-4357-a398-3410e284a58a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:50.023352 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bb26cc-1b84-4357-a398-3410e284a58a admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:50.023636 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bb26cc-1b84-4357-a398-3410e284a58a admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:50.025204 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bb26cc-1b84-4357-a398-3410e284a58a admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:50.026030 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bb26cc-1b84-4357-a398-3410e284a58a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:50.031253 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bb26cc-1b84-4357-a398-3410e284a58a admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:50.055701 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 778/1536] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:29:49 2022] GET /identity/v3/roles => generated 1420 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:29:50.069573 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f5bedf-2223-4430-ae9d-f08643164822 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:50.071280 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783e2273-68ab-41c5-a8e9-ada87cbf7e41 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:50.071576 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783e2273-68ab-41c5-a8e9-ada87cbf7e41 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:50.071854 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783e2273-68ab-41c5-a8e9-ada87cbf7e41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:50.081816 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04966eab-1a17-4b39-864a-833b6bcbe8be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:50.082852 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04966eab-1a17-4b39-864a-833b6bcbe8be admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:50.083341 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04966eab-1a17-4b39-864a-833b6bcbe8be admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:50.086936 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04966eab-1a17-4b39-864a-833b6bcbe8be admin admin] PATH_INFO: `/v3/users` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:50.087725 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04966eab-1a17-4b39-864a-833b6bcbe8be admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:50.089812 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04966eab-1a17-4b39-864a-833b6bcbe8be admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:50.096278 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-783e2273-68ab-41c5-a8e9-ada87cbf7e41 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:29:50.097528 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-783e2273-68ab-41c5-a8e9-ada87cbf7e41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:50.118313 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 779/1537] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:29:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:29:50.118313 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70d174f-8916-4b7b-b0ce-cc9c5894a0ac None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:50.118313 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70d174f-8916-4b7b-b0ce-cc9c5894a0ac None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:50.118313 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70d174f-8916-4b7b-b0ce-cc9c5894a0ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:50.152544 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 759/1538] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Nov 2 07:29:49 2022] POST /identity/v3/users => generated 311 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:29:50.152544 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04966eab-1a17-4b39-864a-833b6bcbe8be admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:50.153353 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-04966eab-1a17-4b39-864a-833b6bcbe8be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:50.157484 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-e70d174f-8916-4b7b-b0ce-cc9c5894a0ac None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:29:50.158838 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-e70d174f-8916-4b7b-b0ce-cc9c5894a0ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:50.167108 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04966eab-1a17-4b39-864a-833b6bcbe8be admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:50.172597 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702174da-2d3e-4074-8e1d-168739c8bacf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:50.173343 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702174da-2d3e-4074-8e1d-168739c8bacf admin admin] REQUEST_METHOD: `PUT` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:50.173730 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 780/1539] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:29:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:29:50.173847 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702174da-2d3e-4074-8e1d-168739c8bacf admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:50.174696 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702174da-2d3e-4074-8e1d-168739c8bacf admin admin] PATH_INFO: `/v3/projects/595ae4b6a60c42d1939420954c75d1eb/users/57b832efd7d94a64beb1e37514a07367/roles/c4bf019fc6c44c71a6c984f6d0bab75f` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:50.181792 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa60107-6b79-47fa-a991-b09f247b28b0 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:50.182081 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa60107-6b79-47fa-a991-b09f247b28b0 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:50.182371 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa60107-6b79-47fa-a991-b09f247b28b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:50.197666 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702174da-2d3e-4074-8e1d-168739c8bacf admin admin] RBAC: Authorizing `identity:create_grant(project_id=595ae4b6a60c42d1939420954c75d1eb, user_id=57b832efd7d94a64beb1e37514a07367, role_id=c4bf019fc6c44c71a6c984f6d0bab75f)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:50.201810 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702174da-2d3e-4074-8e1d-168739c8bacf admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:50.204361 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-6aa60107-6b79-47fa-a991-b09f247b28b0 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:29:50.205638 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-6aa60107-6b79-47fa-a991-b09f247b28b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:50.217019 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 781/1540] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:29:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:29:50.218024 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa60107-6b79-47fa-a991-b09f247b28b0 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:50.222314 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 760/1541] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Nov 2 07:29:50 2022] PUT /identity/v3/projects/595ae4b6a60c42d1939420954c75d1eb/users/57b832efd7d94a64beb1e37514a07367/roles/c4bf019fc6c44c71a6c984f6d0bab75f => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:29:50.224407 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc62ae6-d44c-4db7-8459-ed905978e011 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:50.224897 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc62ae6-d44c-4db7-8459-ed905978e011 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:50.225369 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc62ae6-d44c-4db7-8459-ed905978e011 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:50.232277 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa60107-6b79-47fa-a991-b09f247b28b0 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:50.250968 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-1cc62ae6-d44c-4db7-8459-ed905978e011 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:29:50.252186 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-1cc62ae6-d44c-4db7-8459-ed905978e011 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:50.296449 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:29:50.307230 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e119665-d723-405e-a47f-307bd816e268 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:50.308845 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e119665-d723-405e-a47f-307bd816e268 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:50.309207 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e119665-d723-405e-a47f-307bd816e268 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:50.309547 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e119665-d723-405e-a47f-307bd816e268 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:50.310291 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e119665-d723-405e-a47f-307bd816e268 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:50.312034 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e119665-d723-405e-a47f-307bd816e268 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:50.327009 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 782/1542] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:29:50 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:29:50.328116 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e119665-d723-405e-a47f-307bd816e268 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:50.344066 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e119665-d723-405e-a47f-307bd816e268 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:50.350282 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114193e6-9fca-4616-837e-99830fdfbf20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:50.351251 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114193e6-9fca-4616-837e-99830fdfbf20 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:50.351523 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114193e6-9fca-4616-837e-99830fdfbf20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:50.351796 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114193e6-9fca-4616-837e-99830fdfbf20 admin admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:50.352452 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114193e6-9fca-4616-837e-99830fdfbf20 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:29:50.353166 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114193e6-9fca-4616-837e-99830fdfbf20 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:50.354521 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114193e6-9fca-4616-837e-99830fdfbf20 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:50.358958 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 761/1543] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:29:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:29:50.363611 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc62ae6-d44c-4db7-8459-ed905978e011 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:50.368379 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 783/1544] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Nov 2 07:29:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:29:50.372936 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114193e6-9fca-4616-837e-99830fdfbf20 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:50.375763 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc62ae6-d44c-4db7-8459-ed905978e011 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:50.382923 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114193e6-9fca-4616-837e-99830fdfbf20 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:50.384037 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c49fc8-c467-4182-8444-f282fee150d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:50.384685 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c49fc8-c467-4182-8444-f282fee150d6 admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:50.384971 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c49fc8-c467-4182-8444-f282fee150d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:50.385677 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c49fc8-c467-4182-8444-f282fee150d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:50.385932 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c49fc8-c467-4182-8444-f282fee150d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:50.388191 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c49fc8-c467-4182-8444-f282fee150d6 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:50.389322 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda51359-1a14-4edd-9ece-7bef7b68746b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:50.390137 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda51359-1a14-4edd-9ece-7bef7b68746b admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:50.390505 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda51359-1a14-4edd-9ece-7bef7b68746b admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:50.390957 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda51359-1a14-4edd-9ece-7bef7b68746b admin admin] PATH_INFO: `/v3/projects/a801965f233d41b58b9b3409c28a57ab/users/0eddebbd0f4b4a9497b3771cd1ffa547/roles/c4bf019fc6c44c71a6c984f6d0bab75f` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:50.398944 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 762/1545] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:29:50 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:29:50.403386 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c49fc8-c467-4182-8444-f282fee150d6 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:50.409773 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda51359-1a14-4edd-9ece-7bef7b68746b admin admin] RBAC: Authorizing `identity:create_grant(project_id=a801965f233d41b58b9b3409c28a57ab, user_id=0eddebbd0f4b4a9497b3771cd1ffa547, role_id=c4bf019fc6c44c71a6c984f6d0bab75f)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:50.413003 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda51359-1a14-4edd-9ece-7bef7b68746b admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:50.413933 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c49fc8-c467-4182-8444-f282fee150d6 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:50.424151 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72587b6-e02d-440b-ba4b-a4912b25e352 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:50.425026 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72587b6-e02d-440b-ba4b-a4912b25e352 admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:50.425391 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72587b6-e02d-440b-ba4b-a4912b25e352 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:50.425811 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72587b6-e02d-440b-ba4b-a4912b25e352 admin admin] PATH_INFO: `/v3/domains` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:50.426936 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72587b6-e02d-440b-ba4b-a4912b25e352 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:29:50.427429 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72587b6-e02d-440b-ba4b-a4912b25e352 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:50.430087 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72587b6-e02d-440b-ba4b-a4912b25e352 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:50.440352 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 784/1546] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Nov 2 07:29:50 2022] PUT /identity/v3/projects/a801965f233d41b58b9b3409c28a57ab/users/0eddebbd0f4b4a9497b3771cd1ffa547/roles/c4bf019fc6c44c71a6c984f6d0bab75f => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:29:50.443368 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda51359-1a14-4edd-9ece-7bef7b68746b admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:50.446402 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 763/1547] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Nov 2 07:29:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:29:50.446699 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72587b6-e02d-440b-ba4b-a4912b25e352 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:50.456829 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72587b6-e02d-440b-ba4b-a4912b25e352 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:50.505953 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda51359-1a14-4edd-9ece-7bef7b68746b admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:50.510508 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2307fe96-72f0-4914-ac83-efc46a5b5ab4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:50.511796 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2307fe96-72f0-4914-ac83-efc46a5b5ab4 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:50.511796 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2307fe96-72f0-4914-ac83-efc46a5b5ab4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:50.512289 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2307fe96-72f0-4914-ac83-efc46a5b5ab4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:50.512895 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2307fe96-72f0-4914-ac83-efc46a5b5ab4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:50.514798 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2307fe96-72f0-4914-ac83-efc46a5b5ab4 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:50.515914 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2be0069-5fa0-49c8-8765-1c0a87efd038 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:50.516565 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2be0069-5fa0-49c8-8765-1c0a87efd038 admin admin] REQUEST_METHOD: `PUT` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:50.516843 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2be0069-5fa0-49c8-8765-1c0a87efd038 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:50.517116 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2be0069-5fa0-49c8-8765-1c0a87efd038 admin admin] PATH_INFO: `/v3/projects/595ae4b6a60c42d1939420954c75d1eb/users/57b832efd7d94a64beb1e37514a07367/roles/c4bf019fc6c44c71a6c984f6d0bab75f` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:50.519020 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2be0069-5fa0-49c8-8765-1c0a87efd038 admin admin] RBAC: Authorizing `identity:create_grant(project_id=595ae4b6a60c42d1939420954c75d1eb, user_id=57b832efd7d94a64beb1e37514a07367, role_id=c4bf019fc6c44c71a6c984f6d0bab75f)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:50.521607 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2be0069-5fa0-49c8-8765-1c0a87efd038 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:50.533559 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 764/1548] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Nov 2 07:29:50 2022] PUT /identity/v3/projects/595ae4b6a60c42d1939420954c75d1eb/users/57b832efd7d94a64beb1e37514a07367/roles/c4bf019fc6c44c71a6c984f6d0bab75f => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:29:50.534545 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2be0069-5fa0-49c8-8765-1c0a87efd038 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:50.536398 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:29:50.547283 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 785/1549] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Nov 2 07:29:50 2022] POST /identity/v3/projects => generated 385 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:29:50.547283 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2307fe96-72f0-4914-ac83-efc46a5b5ab4 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:50.599039 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2be0069-5fa0-49c8-8765-1c0a87efd038 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:50.612037 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2307fe96-72f0-4914-ac83-efc46a5b5ab4 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:50.617325 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0f1218-fe8c-4c41-bdc0-8144e3036d6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:50.618046 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0f1218-fe8c-4c41-bdc0-8144e3036d6c admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:50.618452 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0f1218-fe8c-4c41-bdc0-8144e3036d6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:50.619175 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0f1218-fe8c-4c41-bdc0-8144e3036d6c admin admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:50.619870 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0f1218-fe8c-4c41-bdc0-8144e3036d6c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:50.622984 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0f1218-fe8c-4c41-bdc0-8144e3036d6c admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:50.665038 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:29:50.669642 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 786/1550] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Nov 2 07:29:50 2022] POST /identity/v3/projects => generated 393 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:29:50.671024 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463646ee-a97d-492e-abe8-f3dd5b1b139e None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:50.671304 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463646ee-a97d-492e-abe8-f3dd5b1b139e None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:50.671917 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463646ee-a97d-492e-abe8-f3dd5b1b139e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:50.677821 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92df066c-f8fd-4531-9def-ebcd07a52a97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:50.678488 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92df066c-f8fd-4531-9def-ebcd07a52a97 admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:50.678958 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92df066c-f8fd-4531-9def-ebcd07a52a97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:50.679234 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92df066c-f8fd-4531-9def-ebcd07a52a97 admin admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:50.680756 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92df066c-f8fd-4531-9def-ebcd07a52a97 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:50.682377 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92df066c-f8fd-4531-9def-ebcd07a52a97 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:50.692549 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 765/1551] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:29:50 2022] GET /identity/v3/roles => generated 1420 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:29:50.693154 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92df066c-f8fd-4531-9def-ebcd07a52a97 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:50.699911 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-463646ee-a97d-492e-abe8-f3dd5b1b139e None None] MFA Rules not processed for user `57b832efd7d94a64beb1e37514a07367`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:29:50.701162 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-463646ee-a97d-492e-abe8-f3dd5b1b139e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:50.704083 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92df066c-f8fd-4531-9def-ebcd07a52a97 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:50.758720 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452d3491-75dd-4ab5-9ce1-ebacf1b66886 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:50.759240 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452d3491-75dd-4ab5-9ce1-ebacf1b66886 admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:50.759519 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452d3491-75dd-4ab5-9ce1-ebacf1b66886 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:50.759793 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452d3491-75dd-4ab5-9ce1-ebacf1b66886 admin admin] PATH_INFO: `/v3/users` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:50.760433 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452d3491-75dd-4ab5-9ce1-ebacf1b66886 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:50.763134 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452d3491-75dd-4ab5-9ce1-ebacf1b66886 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:50.770099 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 787/1552] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:29:50 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:29:50.771065 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463646ee-a97d-492e-abe8-f3dd5b1b139e None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:50.782809 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463646ee-a97d-492e-abe8-f3dd5b1b139e None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:50.787511 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de14b8c5-1b69-43d2-9593-4f4a72bc9b20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:50.788112 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de14b8c5-1b69-43d2-9593-4f4a72bc9b20 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:50.788389 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de14b8c5-1b69-43d2-9593-4f4a72bc9b20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:50.788657 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de14b8c5-1b69-43d2-9593-4f4a72bc9b20 admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:50.789276 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de14b8c5-1b69-43d2-9593-4f4a72bc9b20 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:50.792866 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 766/1553] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Nov 2 07:29:50 2022] POST /identity/v3/users => generated 293 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:29:50.792866 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452d3491-75dd-4ab5-9ce1-ebacf1b66886 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:50.793225 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de14b8c5-1b69-43d2-9593-4f4a72bc9b20 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:50.801546 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452d3491-75dd-4ab5-9ce1-ebacf1b66886 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:50.807051 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9abe70-a231-4220-8b97-5dc2395f008e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:50.807977 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9abe70-a231-4220-8b97-5dc2395f008e admin admin] REQUEST_METHOD: `PUT` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:50.808743 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9abe70-a231-4220-8b97-5dc2395f008e admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:50.809128 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9abe70-a231-4220-8b97-5dc2395f008e admin admin] PATH_INFO: `/v3/projects/a801965f233d41b58b9b3409c28a57ab/users/0eddebbd0f4b4a9497b3771cd1ffa547/roles/c4bf019fc6c44c71a6c984f6d0bab75f` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:50.811087 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9abe70-a231-4220-8b97-5dc2395f008e admin admin] RBAC: Authorizing `identity:create_grant(project_id=a801965f233d41b58b9b3409c28a57ab, user_id=0eddebbd0f4b4a9497b3771cd1ffa547, role_id=c4bf019fc6c44c71a6c984f6d0bab75f)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:50.813747 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9abe70-a231-4220-8b97-5dc2395f008e admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:50.817204 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 788/1554] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Nov 2 07:29:50 2022] POST /identity/v3/users => generated 297 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:29:50.817204 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de14b8c5-1b69-43d2-9593-4f4a72bc9b20 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:50.819538 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-de14b8c5-1b69-43d2-9593-4f4a72bc9b20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:50.836962 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 767/1555] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Nov 2 07:29:50 2022] PUT /identity/v3/projects/a801965f233d41b58b9b3409c28a57ab/users/0eddebbd0f4b4a9497b3771cd1ffa547/roles/c4bf019fc6c44c71a6c984f6d0bab75f => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:29:50.836962 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9abe70-a231-4220-8b97-5dc2395f008e admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:50.907888 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de14b8c5-1b69-43d2-9593-4f4a72bc9b20 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:50.918805 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:29:50.960302 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9abe70-a231-4220-8b97-5dc2395f008e admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:50.976045 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67020628-af9d-4707-ab27-0a2c620e827e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:50.977669 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67020628-af9d-4707-ab27-0a2c620e827e admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:50.980209 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67020628-af9d-4707-ab27-0a2c620e827e admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:50.994723 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67020628-af9d-4707-ab27-0a2c620e827e admin admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:50.996107 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67020628-af9d-4707-ab27-0a2c620e827e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:50.997695 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67020628-af9d-4707-ab27-0a2c620e827e admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:50.997818 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28374629-9619-4f7a-95ed-1de3f22fedc1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43bb326fb82e4d2c83fb2eaf91860dc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:50.999608 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28374629-9619-4f7a-95ed-1de3f22fedc1 service cinder] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:51.000936 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28374629-9619-4f7a-95ed-1de3f22fedc1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:51.000936 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28374629-9619-4f7a-95ed-1de3f22fedc1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:51.023991 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 768/1556] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:29:50 2022] GET /identity/v3/roles => generated 1420 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:29:51.024937 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67020628-af9d-4707-ab27-0a2c620e827e admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:51.043348 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67020628-af9d-4707-ab27-0a2c620e827e admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:51.056252 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa7cbf6-3509-4f33-8339-836eb5b79f43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:51.057468 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa7cbf6-3509-4f33-8339-836eb5b79f43 admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:51.058129 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa7cbf6-3509-4f33-8339-836eb5b79f43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:51.058559 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa7cbf6-3509-4f33-8339-836eb5b79f43 admin admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:51.059366 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa7cbf6-3509-4f33-8339-836eb5b79f43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:51.060981 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa7cbf6-3509-4f33-8339-836eb5b79f43 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:51.087872 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 769/1557] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:29:51 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:29:51.090282 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa3954f-631c-49eb-86db-17680e78d558 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:51.090846 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa3954f-631c-49eb-86db-17680e78d558 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:51.091819 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa3954f-631c-49eb-86db-17680e78d558 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:51.127395 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28374629-9619-4f7a-95ed-1de3f22fedc1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:51.128984 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28374629-9619-4f7a-95ed-1de3f22fedc1 service cinder] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:51.134615 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-6aa3954f-631c-49eb-86db-17680e78d558 None None] MFA Rules not processed for user `0eddebbd0f4b4a9497b3771cd1ffa547`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:29:51.136080 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-6aa3954f-631c-49eb-86db-17680e78d558 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:51.149519 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 789/1558] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:29:50 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:29:51.150423 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28374629-9619-4f7a-95ed-1de3f22fedc1 service cinder] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:51.175766 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28374629-9619-4f7a-95ed-1de3f22fedc1 service cinder] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:51.193394 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910fbdf8-3fe9-49b4-94f6-8e620b22dd6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:51.194198 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910fbdf8-3fe9-49b4-94f6-8e620b22dd6f admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:51.194653 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910fbdf8-3fe9-49b4-94f6-8e620b22dd6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:51.195084 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910fbdf8-3fe9-49b4-94f6-8e620b22dd6f admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:51.195853 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910fbdf8-3fe9-49b4-94f6-8e620b22dd6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:51.197590 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910fbdf8-3fe9-49b4-94f6-8e620b22dd6f admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:51.200445 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:29:51.225030 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 790/1559] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:29:51 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:29:51.229738 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910fbdf8-3fe9-49b4-94f6-8e620b22dd6f admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:51.243413 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910fbdf8-3fe9-49b4-94f6-8e620b22dd6f admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:51.251346 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3daa52-1b03-4107-b22f-63556cc5bbcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:51.251884 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3daa52-1b03-4107-b22f-63556cc5bbcd admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:51.252164 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3daa52-1b03-4107-b22f-63556cc5bbcd admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:51.252432 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3daa52-1b03-4107-b22f-63556cc5bbcd admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:51.253146 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3daa52-1b03-4107-b22f-63556cc5bbcd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:51.255133 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3daa52-1b03-4107-b22f-63556cc5bbcd admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:51.276101 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 770/1560] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:29:51 2022] POST /identity/v3/auth/tokens => generated 3353 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:29:51.277417 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa3954f-631c-49eb-86db-17680e78d558 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:51.288530 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa3954f-631c-49eb-86db-17680e78d558 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:51.289260 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 791/1561] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:29:51 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:29:51.294183 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ca5d08-3546-4c48-b4ac-4505ac453f20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:51.295980 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ca5d08-3546-4c48-b4ac-4505ac453f20 admin admin] REQUEST_METHOD: `PUT` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:51.296307 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ca5d08-3546-4c48-b4ac-4505ac453f20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:51.296620 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ca5d08-3546-4c48-b4ac-4505ac453f20 admin admin] PATH_INFO: `/v3/projects/b9fcfb25f93948c3888678972b6432a7/users/a7bd7875de244ba6ab08e075f3e0d378/roles/b494fedd00f24f0b84a529e8672a156a` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:51.311443 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3daa52-1b03-4107-b22f-63556cc5bbcd admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:51.319594 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ca5d08-3546-4c48-b4ac-4505ac453f20 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9fcfb25f93948c3888678972b6432a7, user_id=a7bd7875de244ba6ab08e075f3e0d378, role_id=b494fedd00f24f0b84a529e8672a156a)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:51.323539 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ca5d08-3546-4c48-b4ac-4505ac453f20 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:51.340631 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3daa52-1b03-4107-b22f-63556cc5bbcd admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:51.350052 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 771/1562] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Nov 2 07:29:51 2022] PUT /identity/v3/projects/b9fcfb25f93948c3888678972b6432a7/users/a7bd7875de244ba6ab08e075f3e0d378/roles/b494fedd00f24f0b84a529e8672a156a => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:29:51.353446 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ca5d08-3546-4c48-b4ac-4505ac453f20 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:51.439522 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f7436b-6a16-4e57-b971-a0657d3aaf80 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43bb326fb82e4d2c83fb2eaf91860dc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:51.440284 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f7436b-6a16-4e57-b971-a0657d3aaf80 service cinder] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:51.440637 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f7436b-6a16-4e57-b971-a0657d3aaf80 service cinder] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:51.440981 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f7436b-6a16-4e57-b971-a0657d3aaf80 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:51.443516 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-c3f7436b-6a16-4e57-b971-a0657d3aaf80 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:51.488531 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ca5d08-3546-4c48-b4ac-4505ac453f20 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:51.499999 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05b3aec-09bf-4bf8-80ba-d4c24b1c563e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:51.500707 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05b3aec-09bf-4bf8-80ba-d4c24b1c563e admin admin] REQUEST_METHOD: `PUT` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:51.500984 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05b3aec-09bf-4bf8-80ba-d4c24b1c563e admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:51.501256 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05b3aec-09bf-4bf8-80ba-d4c24b1c563e admin admin] PATH_INFO: `/v3/projects/d4ea076280664004aeb1b0d16be1ee75/users/85197edde6aa453ab55854428c8f5973/roles/b494fedd00f24f0b84a529e8672a156a` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:51.524978 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05b3aec-09bf-4bf8-80ba-d4c24b1c563e admin admin] RBAC: Authorizing `identity:create_grant(project_id=d4ea076280664004aeb1b0d16be1ee75, user_id=85197edde6aa453ab55854428c8f5973, role_id=b494fedd00f24f0b84a529e8672a156a)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:51.528959 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05b3aec-09bf-4bf8-80ba-d4c24b1c563e admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:51.539411 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:29:51.550225 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 772/1563] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Nov 2 07:29:51 2022] PUT /identity/v3/projects/d4ea076280664004aeb1b0d16be1ee75/users/85197edde6aa453ab55854428c8f5973/roles/b494fedd00f24f0b84a529e8672a156a => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:29:51.553253 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7becbd-9c89-47b1-803e-53f446340760 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:51.553671 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7becbd-9c89-47b1-803e-53f446340760 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:51.554074 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7becbd-9c89-47b1-803e-53f446340760 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:51.555974 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f7436b-6a16-4e57-b971-a0657d3aaf80 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:51.557660 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f7436b-6a16-4e57-b971-a0657d3aaf80 service cinder] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:51.580140 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 792/1564] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:29:51 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:29:51.583955 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47800b34-a628-4fe1-8082-f58e54ae3708 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:51.584238 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47800b34-a628-4fe1-8082-f58e54ae3708 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:51.584509 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47800b34-a628-4fe1-8082-f58e54ae3708 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:51.598848 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-9b7becbd-9c89-47b1-803e-53f446340760 None None] MFA Rules not processed for user `a7bd7875de244ba6ab08e075f3e0d378`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:29:51.601037 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-9b7becbd-9c89-47b1-803e-53f446340760 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:51.646593 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-47800b34-a628-4fe1-8082-f58e54ae3708 None None] MFA Rules not processed for user `85197edde6aa453ab55854428c8f5973`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:29:51.649206 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-47800b34-a628-4fe1-8082-f58e54ae3708 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:51.772153 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 773/1565] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:29:51 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:29:51.773245 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7becbd-9c89-47b1-803e-53f446340760 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:51.774895 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-9b7becbd-9c89-47b1-803e-53f446340760 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:51.816942 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 793/1566] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:29:51 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:29:51.816942 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47800b34-a628-4fe1-8082-f58e54ae3708 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:51.848717 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:29:51.879637 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7becbd-9c89-47b1-803e-53f446340760 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:51.886667 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a9d228-3310-4c4b-a3a5-8568e0ce98c4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43bb326fb82e4d2c83fb2eaf91860dc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:51.887242 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a9d228-3310-4c4b-a3a5-8568e0ce98c4 service cinder] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:51.887524 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a9d228-3310-4c4b-a3a5-8568e0ce98c4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:51.887796 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a9d228-3310-4c4b-a3a5-8568e0ce98c4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:51.927839 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47800b34-a628-4fe1-8082-f58e54ae3708 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:51.937016 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cdddcf-7a5c-47b5-9568-f063f3b67f65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:51.938353 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cdddcf-7a5c-47b5-9568-f063f3b67f65 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:51.938974 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cdddcf-7a5c-47b5-9568-f063f3b67f65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:51.939254 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cdddcf-7a5c-47b5-9568-f063f3b67f65 admin admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:51.939899 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cdddcf-7a5c-47b5-9568-f063f3b67f65 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:51.942585 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cdddcf-7a5c-47b5-9568-f063f3b67f65 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:51.976580 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a9d228-3310-4c4b-a3a5-8568e0ce98c4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:51.978606 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a9d228-3310-4c4b-a3a5-8568e0ce98c4 service cinder] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:51.982442 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 794/1567] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Nov 2 07:29:51 2022] POST /identity/v3/projects => generated 385 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:29:51.983281 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cdddcf-7a5c-47b5-9568-f063f3b67f65 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:51.995809 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 774/1568] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:29:51 2022] GET /identity/v3/auth/tokens => generated 3353 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:29:51.997504 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a9d228-3310-4c4b-a3a5-8568e0ce98c4 service cinder] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:52.027791 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a9d228-3310-4c4b-a3a5-8568e0ce98c4 service cinder] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:52.118010 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cdddcf-7a5c-47b5-9568-f063f3b67f65 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:52.125136 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a469f7-20e5-4113-9ee8-25b9f7e4c91b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:52.126022 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a469f7-20e5-4113-9ee8-25b9f7e4c91b admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:52.126217 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a469f7-20e5-4113-9ee8-25b9f7e4c91b admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:52.127594 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a469f7-20e5-4113-9ee8-25b9f7e4c91b admin admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:52.128269 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a469f7-20e5-4113-9ee8-25b9f7e4c91b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:52.133551 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a469f7-20e5-4113-9ee8-25b9f7e4c91b admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:52.166598 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:29:52.222638 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 795/1569] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Nov 2 07:29:51 2022] POST /identity/v3/projects => generated 391 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:29:52.229064 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a469f7-20e5-4113-9ee8-25b9f7e4c91b admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:52.363177 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ff1c7e-15b8-481d-be57-2abb3c8f04df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:52.364452 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ff1c7e-15b8-481d-be57-2abb3c8f04df admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:52.364869 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ff1c7e-15b8-481d-be57-2abb3c8f04df admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:52.365273 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ff1c7e-15b8-481d-be57-2abb3c8f04df admin admin] PATH_INFO: `/v3/users` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:52.366163 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ff1c7e-15b8-481d-be57-2abb3c8f04df admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:52.371184 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ff1c7e-15b8-481d-be57-2abb3c8f04df admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:52.402577 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a469f7-20e5-4113-9ee8-25b9f7e4c91b admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:52.422944 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3a07ee-3dd6-41f8-be09-08928dc6162e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:52.427319 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3a07ee-3dd6-41f8-be09-08928dc6162e admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:52.427852 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3a07ee-3dd6-41f8-be09-08928dc6162e admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:52.428460 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3a07ee-3dd6-41f8-be09-08928dc6162e admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:52.429440 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3a07ee-3dd6-41f8-be09-08928dc6162e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:52.431042 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 775/1570] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Nov 2 07:29:51 2022] POST /identity/v3/users => generated 299 bytes in 435 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:29:52.433941 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ff1c7e-15b8-481d-be57-2abb3c8f04df admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:52.435448 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3a07ee-3dd6-41f8-be09-08928dc6162e admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:52.486347 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 796/1571] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Nov 2 07:29:52 2022] POST /identity/v3/users => generated 302 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:29:52.487300 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3a07ee-3dd6-41f8-be09-08928dc6162e admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:52.519395 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3a07ee-3dd6-41f8-be09-08928dc6162e admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:52.536218 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3841b57a-a68f-49d1-8107-12114679e1be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:52.539946 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3841b57a-a68f-49d1-8107-12114679e1be admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:52.542732 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3841b57a-a68f-49d1-8107-12114679e1be admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:52.543701 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3841b57a-a68f-49d1-8107-12114679e1be admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:52.545394 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3841b57a-a68f-49d1-8107-12114679e1be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:52.551635 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3841b57a-a68f-49d1-8107-12114679e1be admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:52.574723 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ff1c7e-15b8-481d-be57-2abb3c8f04df admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:52.577869 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 797/1572] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:29:52 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:29:52.586885 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3841b57a-a68f-49d1-8107-12114679e1be admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:52.604556 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3841b57a-a68f-49d1-8107-12114679e1be admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:52.619993 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfda12e-a1c8-46e1-9e05-d72e40142d4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:52.621270 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:29:52.624369 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfda12e-a1c8-46e1-9e05-d72e40142d4a admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:52.624757 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfda12e-a1c8-46e1-9e05-d72e40142d4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:52.624928 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfda12e-a1c8-46e1-9e05-d72e40142d4a admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:52.626964 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfda12e-a1c8-46e1-9e05-d72e40142d4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:52.631420 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfda12e-a1c8-46e1-9e05-d72e40142d4a admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:52.659992 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 798/1573] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:29:52 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:29:52.663815 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfda12e-a1c8-46e1-9e05-d72e40142d4a admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:52.704027 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfda12e-a1c8-46e1-9e05-d72e40142d4a admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:52.717816 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a74c75-f2ff-4f64-85e8-4df842b5edff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:52.718526 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a74c75-f2ff-4f64-85e8-4df842b5edff admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:52.719307 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97663378-f1bb-4447-b09b-664e5148ec27 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43bb326fb82e4d2c83fb2eaf91860dc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:52.719935 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97663378-f1bb-4447-b09b-664e5148ec27 service cinder] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:52.720219 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97663378-f1bb-4447-b09b-664e5148ec27 service cinder] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:52.728486 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97663378-f1bb-4447-b09b-664e5148ec27 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:52.728849 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a74c75-f2ff-4f64-85e8-4df842b5edff admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:52.728849 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a74c75-f2ff-4f64-85e8-4df842b5edff admin admin] PATH_INFO: `/v3/projects/5fde67b668354c5f8129ce79eb73b177/users/85fab4413c034c0e86fa3ec4d161c77f/roles/c4bf019fc6c44c71a6c984f6d0bab75f` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:52.728983 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-97663378-f1bb-4447-b09b-664e5148ec27 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:52.759915 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a74c75-f2ff-4f64-85e8-4df842b5edff admin admin] RBAC: Authorizing `identity:create_grant(project_id=5fde67b668354c5f8129ce79eb73b177, user_id=85fab4413c034c0e86fa3ec4d161c77f, role_id=c4bf019fc6c44c71a6c984f6d0bab75f)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:52.767923 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a74c75-f2ff-4f64-85e8-4df842b5edff admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:52.798528 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 799/1574] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Nov 2 07:29:52 2022] PUT /identity/v3/projects/5fde67b668354c5f8129ce79eb73b177/users/85fab4413c034c0e86fa3ec4d161c77f/roles/c4bf019fc6c44c71a6c984f6d0bab75f => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:29:52.799556 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a74c75-f2ff-4f64-85e8-4df842b5edff admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:52.840680 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97663378-f1bb-4447-b09b-664e5148ec27 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:52.842139 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97663378-f1bb-4447-b09b-664e5148ec27 service cinder] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:52.858939 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 776/1575] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:29:52 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:29:52.863211 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97663378-f1bb-4447-b09b-664e5148ec27 service cinder] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:52.894714 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97663378-f1bb-4447-b09b-664e5148ec27 service cinder] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:52.906591 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74b77a1-4bcd-434b-a919-300227d95b3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:52.908755 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74b77a1-4bcd-434b-a919-300227d95b3e admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:52.909045 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74b77a1-4bcd-434b-a919-300227d95b3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:52.909316 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74b77a1-4bcd-434b-a919-300227d95b3e admin admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:52.910040 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74b77a1-4bcd-434b-a919-300227d95b3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:52.913214 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74b77a1-4bcd-434b-a919-300227d95b3e admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:52.917873 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a74c75-f2ff-4f64-85e8-4df842b5edff admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:52.931690 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aac4b76-9eb3-4f6a-8234-94dc3e45e767 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:52.932374 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aac4b76-9eb3-4f6a-8234-94dc3e45e767 admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:52.932653 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aac4b76-9eb3-4f6a-8234-94dc3e45e767 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:52.933549 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aac4b76-9eb3-4f6a-8234-94dc3e45e767 admin admin] PATH_INFO: `/v3/projects/1064d39b9ed94a168e5c5392842b1315/users/7ddc29add40446b2a8ff2abe7c77fd5d/roles/c4bf019fc6c44c71a6c984f6d0bab75f` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:52.953722 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 777/1576] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:29:52 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:29:52.954086 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74b77a1-4bcd-434b-a919-300227d95b3e admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:52.978171 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aac4b76-9eb3-4f6a-8234-94dc3e45e767 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1064d39b9ed94a168e5c5392842b1315, user_id=7ddc29add40446b2a8ff2abe7c77fd5d, role_id=c4bf019fc6c44c71a6c984f6d0bab75f)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:52.982071 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74b77a1-4bcd-434b-a919-300227d95b3e admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:52.986962 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aac4b76-9eb3-4f6a-8234-94dc3e45e767 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:52.994787 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62045c5f-cc9c-4325-95ad-7cdcf4260aed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:52.996964 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62045c5f-cc9c-4325-95ad-7cdcf4260aed admin admin] REQUEST_METHOD: `PUT` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:52.997260 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62045c5f-cc9c-4325-95ad-7cdcf4260aed admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:52.997511 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62045c5f-cc9c-4325-95ad-7cdcf4260aed admin admin] PATH_INFO: `/v3/projects/5fde67b668354c5f8129ce79eb73b177/users/85fab4413c034c0e86fa3ec4d161c77f/roles/c4bf019fc6c44c71a6c984f6d0bab75f` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:53.001346 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62045c5f-cc9c-4325-95ad-7cdcf4260aed admin admin] RBAC: Authorizing `identity:create_grant(project_id=5fde67b668354c5f8129ce79eb73b177, user_id=85fab4413c034c0e86fa3ec4d161c77f, role_id=c4bf019fc6c44c71a6c984f6d0bab75f)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:53.006337 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62045c5f-cc9c-4325-95ad-7cdcf4260aed admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:53.021703 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 800/1577] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Nov 2 07:29:52 2022] PUT /identity/v3/projects/1064d39b9ed94a168e5c5392842b1315/users/7ddc29add40446b2a8ff2abe7c77fd5d/roles/c4bf019fc6c44c71a6c984f6d0bab75f => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:29:53.031759 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 778/1578] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Nov 2 07:29:52 2022] PUT /identity/v3/projects/5fde67b668354c5f8129ce79eb73b177/users/85fab4413c034c0e86fa3ec4d161c77f/roles/c4bf019fc6c44c71a6c984f6d0bab75f => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:29:53.035969 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aac4b76-9eb3-4f6a-8234-94dc3e45e767 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:53.061744 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee21ecd-ab59-45ec-9c66-c3a4a0837085 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:53.062951 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee21ecd-ab59-45ec-9c66-c3a4a0837085 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:53.065621 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee21ecd-ab59-45ec-9c66-c3a4a0837085 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:53.095416 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:29:53.146192 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-2ee21ecd-ab59-45ec-9c66-c3a4a0837085 None None] MFA Rules not processed for user `85fab4413c034c0e86fa3ec4d161c77f`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:29:53.150288 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-2ee21ecd-ab59-45ec-9c66-c3a4a0837085 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:53.207055 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aac4b76-9eb3-4f6a-8234-94dc3e45e767 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:53.284960 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 779/1579] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:29:53 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:29:53.319587 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee21ecd-ab59-45ec-9c66-c3a4a0837085 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:53.332941 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af0ccee-79eb-490d-bb48-63cc87bc98ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:53.333749 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af0ccee-79eb-490d-bb48-63cc87bc98ee admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:53.334136 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af0ccee-79eb-490d-bb48-63cc87bc98ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:53.334836 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af0ccee-79eb-490d-bb48-63cc87bc98ee admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:53.335531 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af0ccee-79eb-490d-bb48-63cc87bc98ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:53.339040 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af0ccee-79eb-490d-bb48-63cc87bc98ee admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:53.359325 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 801/1580] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:29:53 2022] GET /identity/v3/roles => generated 1420 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:29:53.363001 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af0ccee-79eb-490d-bb48-63cc87bc98ee admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:53.373493 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:29:53.381071 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af0ccee-79eb-490d-bb48-63cc87bc98ee admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:53.391536 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6770696-d3cf-4041-9b4d-12a4057d6f62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:53.392873 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6770696-d3cf-4041-9b4d-12a4057d6f62 admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:53.393156 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6770696-d3cf-4041-9b4d-12a4057d6f62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:53.393493 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6770696-d3cf-4041-9b4d-12a4057d6f62 admin admin] PATH_INFO: `/v3/projects/1064d39b9ed94a168e5c5392842b1315/users/7ddc29add40446b2a8ff2abe7c77fd5d/roles/c4bf019fc6c44c71a6c984f6d0bab75f` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:53.395861 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6770696-d3cf-4041-9b4d-12a4057d6f62 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1064d39b9ed94a168e5c5392842b1315, user_id=7ddc29add40446b2a8ff2abe7c77fd5d, role_id=c4bf019fc6c44c71a6c984f6d0bab75f)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:53.398814 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6770696-d3cf-4041-9b4d-12a4057d6f62 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:53.399125 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee21ecd-ab59-45ec-9c66-c3a4a0837085 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:53.412602 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81b2d26-962e-4930-93f8-bd24cad8935c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43bb326fb82e4d2c83fb2eaf91860dc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:53.413645 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81b2d26-962e-4930-93f8-bd24cad8935c service cinder] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:53.414121 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81b2d26-962e-4930-93f8-bd24cad8935c service cinder] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:53.414730 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 802/1581] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Nov 2 07:29:53 2022] PUT /identity/v3/projects/1064d39b9ed94a168e5c5392842b1315/users/7ddc29add40446b2a8ff2abe7c77fd5d/roles/c4bf019fc6c44c71a6c984f6d0bab75f => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 07:29:53.415269 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81b2d26-962e-4930-93f8-bd24cad8935c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:53.427447 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cee312-99af-4b99-974f-f3d3ef2512fe None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:53.427732 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cee312-99af-4b99-974f-f3d3ef2512fe None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:53.428006 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cee312-99af-4b99-974f-f3d3ef2512fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:53.442646 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81b2d26-962e-4930-93f8-bd24cad8935c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:53.444310 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81b2d26-962e-4930-93f8-bd24cad8935c service cinder] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:53.467349 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-10cee312-99af-4b99-974f-f3d3ef2512fe None None] MFA Rules not processed for user `7ddc29add40446b2a8ff2abe7c77fd5d`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:29:53.468969 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-10cee312-99af-4b99-974f-f3d3ef2512fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:53.481953 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 780/1582] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:29:53 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:29:53.563777 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 803/1583] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:29:53 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:29:53.633167 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81b2d26-962e-4930-93f8-bd24cad8935c service cinder] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:53.643618 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81b2d26-962e-4930-93f8-bd24cad8935c service cinder] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:53.708319 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c91dad-632a-426c-bd65-8d6be866b7b3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43bb326fb82e4d2c83fb2eaf91860dc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:53.709100 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c91dad-632a-426c-bd65-8d6be866b7b3 service cinder] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:53.709483 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c91dad-632a-426c-bd65-8d6be866b7b3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:53.709878 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c91dad-632a-426c-bd65-8d6be866b7b3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:53.721628 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c91dad-632a-426c-bd65-8d6be866b7b3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:53.723413 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c91dad-632a-426c-bd65-8d6be866b7b3 service cinder] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:53.749876 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 781/1584] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:29:53 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:29:53.847805 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cee312-99af-4b99-974f-f3d3ef2512fe None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:53.863534 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cee312-99af-4b99-974f-f3d3ef2512fe None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:53.876193 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123df89f-19c5-4a7b-9184-c4273e065f80 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43bb326fb82e4d2c83fb2eaf91860dc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:53.876830 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123df89f-19c5-4a7b-9184-c4273e065f80 service cinder] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:53.877105 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123df89f-19c5-4a7b-9184-c4273e065f80 service cinder] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:53.893759 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123df89f-19c5-4a7b-9184-c4273e065f80 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:53.963998 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:29:54.008475 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123df89f-19c5-4a7b-9184-c4273e065f80 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:54.009923 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123df89f-19c5-4a7b-9184-c4273e065f80 service cinder] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:54.021818 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 804/1585] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:29:53 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:29:54.498751 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c91dad-632a-426c-bd65-8d6be866b7b3 service cinder] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:54.634039 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c91dad-632a-426c-bd65-8d6be866b7b3 service cinder] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:54.645822 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d447a3ab-de10-44de-a3ad-239af20570e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:54.652258 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d447a3ab-de10-44de-a3ad-239af20570e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:54.652649 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d447a3ab-de10-44de-a3ad-239af20570e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:54.653033 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d447a3ab-de10-44de-a3ad-239af20570e9 admin admin] PATH_INFO: `/v3/users/a7bd7875de244ba6ab08e075f3e0d378` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:54.654507 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d447a3ab-de10-44de-a3ad-239af20570e9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a7bd7875de244ba6ab08e075f3e0d378)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:54.656567 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d447a3ab-de10-44de-a3ad-239af20570e9 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:54.727141 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-d447a3ab-de10-44de-a3ad-239af20570e9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a7bd7875de244ba6ab08e075f3e0d378'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:54.738264 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-d447a3ab-de10-44de-a3ad-239af20570e9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a7bd7875de244ba6ab08e075f3e0d378'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:54.739523 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-d447a3ab-de10-44de-a3ad-239af20570e9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a7bd7875de244ba6ab08e075f3e0d378'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:54.750680 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-d447a3ab-de10-44de-a3ad-239af20570e9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a7bd7875de244ba6ab08e075f3e0d378'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:54.789570 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-d447a3ab-de10-44de-a3ad-239af20570e9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a7bd7875de244ba6ab08e075f3e0d378'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:54.807578 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 782/1586] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Nov 2 07:29:54 2022] DELETE /identity/v3/users/a7bd7875de244ba6ab08e075f3e0d378 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:29:54.812894 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ffeaeb-6284-4735-8e23-767a9e24ddba None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:54.813366 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ffeaeb-6284-4735-8e23-767a9e24ddba None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:54.813693 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ffeaeb-6284-4735-8e23-767a9e24ddba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:54.836110 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-c7ffeaeb-6284-4735-8e23-767a9e24ddba None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:29:54.837438 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-c7ffeaeb-6284-4735-8e23-767a9e24ddba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:54.934406 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 805/1587] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:29:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:29:55.168186 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d447a3ab-de10-44de-a3ad-239af20570e9 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:55.170322 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-d447a3ab-de10-44de-a3ad-239af20570e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:55.178837 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:29:55.251755 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d447a3ab-de10-44de-a3ad-239af20570e9 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:55.277778 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efadce1a-8c4d-4217-968c-cb8a1277cc69 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3ec6ecb85f94ef59762877d9ea33ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:55.278984 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efadce1a-8c4d-4217-968c-cb8a1277cc69 service neutron] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:55.281057 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efadce1a-8c4d-4217-968c-cb8a1277cc69 service neutron] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:55.281447 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efadce1a-8c4d-4217-968c-cb8a1277cc69 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:55.300416 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efadce1a-8c4d-4217-968c-cb8a1277cc69 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:55.304059 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efadce1a-8c4d-4217-968c-cb8a1277cc69 service neutron] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:55.330703 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 783/1588] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:29:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:29:55.848386 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ffeaeb-6284-4735-8e23-767a9e24ddba None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:55.851950 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-c7ffeaeb-6284-4735-8e23-767a9e24ddba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:55.865696 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ffeaeb-6284-4735-8e23-767a9e24ddba None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:55.871057 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e408a1-d78b-4d03-866b-1c2a419524d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:55.871696 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e408a1-d78b-4d03-866b-1c2a419524d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:55.872447 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e408a1-d78b-4d03-866b-1c2a419524d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:55.872726 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e408a1-d78b-4d03-866b-1c2a419524d4 admin admin] PATH_INFO: `/v3/projects/b9fcfb25f93948c3888678972b6432a7` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:55.873850 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e408a1-d78b-4d03-866b-1c2a419524d4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b9fcfb25f93948c3888678972b6432a7)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:55.875772 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e408a1-d78b-4d03-866b-1c2a419524d4 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:55.926616 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-a9e408a1-d78b-4d03-866b-1c2a419524d4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b9fcfb25f93948c3888678972b6432a7', 'request_id': 'req-a9e408a1-d78b-4d03-866b-1c2a419524d4'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:55.933224 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-a9e408a1-d78b-4d03-866b-1c2a419524d4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b9fcfb25f93948c3888678972b6432a7', 'request_id': 'req-a9e408a1-d78b-4d03-866b-1c2a419524d4'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:55.938982 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-a9e408a1-d78b-4d03-866b-1c2a419524d4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b9fcfb25f93948c3888678972b6432a7', 'request_id': 'req-a9e408a1-d78b-4d03-866b-1c2a419524d4'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:55.952610 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-a9e408a1-d78b-4d03-866b-1c2a419524d4 admin admin] The token cache is being invalidate because project b9fcfb25f93948c3888678972b6432a7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:29:55.952897 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-a9e408a1-d78b-4d03-866b-1c2a419524d4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:55.953524 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-a9e408a1-d78b-4d03-866b-1c2a419524d4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:55.955057 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 806/1589] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Nov 2 07:29:55 2022] DELETE /identity/v3/projects/b9fcfb25f93948c3888678972b6432a7 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 07:29:55.958818 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efadce1a-8c4d-4217-968c-cb8a1277cc69 service neutron] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:55.969236 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efadce1a-8c4d-4217-968c-cb8a1277cc69 service neutron] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:56.025804 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b466a32-95a6-426c-a3f3-6b0c0f98f497 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:56.027227 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b466a32-95a6-426c-a3f3-6b0c0f98f497 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:56.027574 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b466a32-95a6-426c-a3f3-6b0c0f98f497 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:56.027908 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b466a32-95a6-426c-a3f3-6b0c0f98f497 admin admin] PATH_INFO: `/v3/users/85fab4413c034c0e86fa3ec4d161c77f` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:56.029157 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b466a32-95a6-426c-a3f3-6b0c0f98f497 admin admin] RBAC: Authorizing `identity:delete_user(user_id=85fab4413c034c0e86fa3ec4d161c77f)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:56.031311 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b466a32-95a6-426c-a3f3-6b0c0f98f497 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:56.077884 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-7b466a32-95a6-426c-a3f3-6b0c0f98f497 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '85fab4413c034c0e86fa3ec4d161c77f'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:56.085754 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-7b466a32-95a6-426c-a3f3-6b0c0f98f497 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '85fab4413c034c0e86fa3ec4d161c77f'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:56.086446 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-7b466a32-95a6-426c-a3f3-6b0c0f98f497 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '85fab4413c034c0e86fa3ec4d161c77f'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:56.094312 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-7b466a32-95a6-426c-a3f3-6b0c0f98f497 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '85fab4413c034c0e86fa3ec4d161c77f'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:56.107727 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-7b466a32-95a6-426c-a3f3-6b0c0f98f497 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '85fab4413c034c0e86fa3ec4d161c77f'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:56.113547 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 784/1590] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Nov 2 07:29:55 2022] DELETE /identity/v3/users/85fab4413c034c0e86fa3ec4d161c77f => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:29:56.394507 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e408a1-d78b-4d03-866b-1c2a419524d4 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:56.396010 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-a9e408a1-d78b-4d03-866b-1c2a419524d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:56.411144 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e408a1-d78b-4d03-866b-1c2a419524d4 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:56.468278 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:29:56.483563 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5b6b86-0061-4ef3-b1b8-4e23c93244f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:56.484351 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5b6b86-0061-4ef3-b1b8-4e23c93244f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:56.484657 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5b6b86-0061-4ef3-b1b8-4e23c93244f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:56.484932 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5b6b86-0061-4ef3-b1b8-4e23c93244f8 admin admin] PATH_INFO: `/v3/projects/5fde67b668354c5f8129ce79eb73b177` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:56.486520 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5b6b86-0061-4ef3-b1b8-4e23c93244f8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5fde67b668354c5f8129ce79eb73b177)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:56.488398 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5b6b86-0061-4ef3-b1b8-4e23c93244f8 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:56.536659 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-9e5b6b86-0061-4ef3-b1b8-4e23c93244f8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5fde67b668354c5f8129ce79eb73b177', 'request_id': 'req-9e5b6b86-0061-4ef3-b1b8-4e23c93244f8'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:56.542754 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-9e5b6b86-0061-4ef3-b1b8-4e23c93244f8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5fde67b668354c5f8129ce79eb73b177', 'request_id': 'req-9e5b6b86-0061-4ef3-b1b8-4e23c93244f8'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:56.548243 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-9e5b6b86-0061-4ef3-b1b8-4e23c93244f8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5fde67b668354c5f8129ce79eb73b177', 'request_id': 'req-9e5b6b86-0061-4ef3-b1b8-4e23c93244f8'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:56.560892 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-9e5b6b86-0061-4ef3-b1b8-4e23c93244f8 admin admin] The token cache is being invalidate because project 5fde67b668354c5f8129ce79eb73b177 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:29:56.561176 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-9e5b6b86-0061-4ef3-b1b8-4e23c93244f8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:56.561795 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-9e5b6b86-0061-4ef3-b1b8-4e23c93244f8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:56.563315 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 807/1591] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Nov 2 07:29:56 2022] DELETE /identity/v3/projects/5fde67b668354c5f8129ce79eb73b177 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:29:56.568138 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2c4b27-1656-42a4-82c2-12a54289a96b None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:56.568424 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2c4b27-1656-42a4-82c2-12a54289a96b None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:56.568696 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2c4b27-1656-42a4-82c2-12a54289a96b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:56.587990 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-0a2c4b27-1656-42a4-82c2-12a54289a96b None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:29:56.589197 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-0a2c4b27-1656-42a4-82c2-12a54289a96b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:56.607020 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:29:56.659290 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 785/1592] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:29:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:29:56.668730 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa331ee-3e19-4f23-be3a-b9c4ca1d2697 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:56.669016 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa331ee-3e19-4f23-be3a-b9c4ca1d2697 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:56.669316 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa331ee-3e19-4f23-be3a-b9c4ca1d2697 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:56.691881 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-efa331ee-3e19-4f23-be3a-b9c4ca1d2697 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:29:56.693081 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-efa331ee-3e19-4f23-be3a-b9c4ca1d2697 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:56.701481 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 808/1593] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:29:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:29:56.705374 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2c4b27-1656-42a4-82c2-12a54289a96b None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:56.715607 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2c4b27-1656-42a4-82c2-12a54289a96b None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:56.720305 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1be32b-b2d9-4a03-8fe8-350229ab77c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:56.720913 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1be32b-b2d9-4a03-8fe8-350229ab77c8 admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:56.721200 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1be32b-b2d9-4a03-8fe8-350229ab77c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:56.721476 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1be32b-b2d9-4a03-8fe8-350229ab77c8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:56.722340 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1be32b-b2d9-4a03-8fe8-350229ab77c8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:29:56.722674 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1be32b-b2d9-4a03-8fe8-350229ab77c8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:56.724273 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1be32b-b2d9-4a03-8fe8-350229ab77c8 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:56.737906 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 786/1594] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Nov 2 07:29:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:29:56.742704 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa331ee-3e19-4f23-be3a-b9c4ca1d2697 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:56.752533 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa331ee-3e19-4f23-be3a-b9c4ca1d2697 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:56.757626 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb3efb1-1f36-4378-84c4-f8aa21599d80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:56.758223 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb3efb1-1f36-4378-84c4-f8aa21599d80 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:56.758537 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb3efb1-1f36-4378-84c4-f8aa21599d80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:56.758811 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb3efb1-1f36-4378-84c4-f8aa21599d80 admin admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:56.759446 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb3efb1-1f36-4378-84c4-f8aa21599d80 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:56.761444 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb3efb1-1f36-4378-84c4-f8aa21599d80 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:56.794930 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 809/1595] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Nov 2 07:29:56 2022] POST /identity/v3/projects => generated 375 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:29:56.796022 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1be32b-b2d9-4a03-8fe8-350229ab77c8 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:56.807213 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1be32b-b2d9-4a03-8fe8-350229ab77c8 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:56.878270 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c0e927-19cd-4d16-9a70-5c7f0672efeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:56.878918 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c0e927-19cd-4d16-9a70-5c7f0672efeb admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:56.879195 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c0e927-19cd-4d16-9a70-5c7f0672efeb admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:56.879464 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c0e927-19cd-4d16-9a70-5c7f0672efeb admin admin] PATH_INFO: `/v3/users` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:56.880098 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c0e927-19cd-4d16-9a70-5c7f0672efeb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:56.882058 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c0e927-19cd-4d16-9a70-5c7f0672efeb admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:56.904853 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 787/1596] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Nov 2 07:29:56 2022] POST /identity/v3/users => generated 288 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:29:56.909228 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb3efb1-1f36-4378-84c4-f8aa21599d80 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:56.919698 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb3efb1-1f36-4378-84c4-f8aa21599d80 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:56.924556 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adb5128-2756-4174-befc-94876362bb2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:56.925156 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adb5128-2756-4174-befc-94876362bb2f admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:56.925432 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adb5128-2756-4174-befc-94876362bb2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:56.925698 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adb5128-2756-4174-befc-94876362bb2f admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:56.926325 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4adb5128-2756-4174-befc-94876362bb2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:56.927974 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4adb5128-2756-4174-befc-94876362bb2f admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:56.939496 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 810/1597] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:29:56 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:29:56.943312 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c0e927-19cd-4d16-9a70-5c7f0672efeb admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:56.967305 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c0e927-19cd-4d16-9a70-5c7f0672efeb admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:56.972930 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363e9b13-fd04-4dd0-bacd-49e71be7a340 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:56.973613 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363e9b13-fd04-4dd0-bacd-49e71be7a340 admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:56.973961 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363e9b13-fd04-4dd0-bacd-49e71be7a340 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:56.974301 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363e9b13-fd04-4dd0-bacd-49e71be7a340 admin admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:56.975037 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363e9b13-fd04-4dd0-bacd-49e71be7a340 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:56.976755 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363e9b13-fd04-4dd0-bacd-49e71be7a340 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:56.998609 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 788/1598] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:29:56 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:29:56.999438 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adb5128-2756-4174-befc-94876362bb2f admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:57.006307 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adb5128-2756-4174-befc-94876362bb2f admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:57.010934 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2501e1b8-133e-4fc8-99df-cca3eb73f854 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:57.011586 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2501e1b8-133e-4fc8-99df-cca3eb73f854 admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:57.011864 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2501e1b8-133e-4fc8-99df-cca3eb73f854 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:57.012134 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2501e1b8-133e-4fc8-99df-cca3eb73f854 admin admin] PATH_INFO: `/v3/projects/31e763668e014b6eb992984f254722c1/users/9bcb3c4382bc4720a91b094cfee1dd78/roles/b494fedd00f24f0b84a529e8672a156a` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:57.027096 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2501e1b8-133e-4fc8-99df-cca3eb73f854 admin admin] RBAC: Authorizing `identity:create_grant(project_id=31e763668e014b6eb992984f254722c1, user_id=9bcb3c4382bc4720a91b094cfee1dd78, role_id=b494fedd00f24f0b84a529e8672a156a)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:57.030112 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2501e1b8-133e-4fc8-99df-cca3eb73f854 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:57.043934 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 811/1599] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Nov 2 07:29:56 2022] PUT /identity/v3/projects/31e763668e014b6eb992984f254722c1/users/9bcb3c4382bc4720a91b094cfee1dd78/roles/b494fedd00f24f0b84a529e8672a156a => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 07:29:57.054322 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b71110-5b21-481b-a38c-6d2deb3ee0cc None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:57.054775 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b71110-5b21-481b-a38c-6d2deb3ee0cc None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:57.055116 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b71110-5b21-481b-a38c-6d2deb3ee0cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:57.084249 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-78b71110-5b21-481b-a38c-6d2deb3ee0cc None None] MFA Rules not processed for user `9bcb3c4382bc4720a91b094cfee1dd78`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:29:57.085551 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-78b71110-5b21-481b-a38c-6d2deb3ee0cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:57.152499 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 789/1600] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:29:57 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:29:57.163620 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2501e1b8-133e-4fc8-99df-cca3eb73f854 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:57.165422 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-2501e1b8-133e-4fc8-99df-cca3eb73f854 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:57.231240 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2501e1b8-133e-4fc8-99df-cca3eb73f854 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:57.234941 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18daafc-c827-46ce-8c49-1a1d0f2ff77e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43bb326fb82e4d2c83fb2eaf91860dc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:57.235571 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18daafc-c827-46ce-8c49-1a1d0f2ff77e service cinder] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:57.235851 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18daafc-c827-46ce-8c49-1a1d0f2ff77e service cinder] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:57.236118 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18daafc-c827-46ce-8c49-1a1d0f2ff77e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:57.244482 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18daafc-c827-46ce-8c49-1a1d0f2ff77e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:57.246305 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18daafc-c827-46ce-8c49-1a1d0f2ff77e service cinder] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:57.254939 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 812/1601] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:29:57 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:29:58.004909 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ef8107-3c1e-4927-8be8-86ba7757e637 None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:58.005447 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ef8107-3c1e-4927-8be8-86ba7757e637 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:58.006169 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ef8107-3c1e-4927-8be8-86ba7757e637 None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:58.008924 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 790/1602] 10.222.0.18 () {54 vars in 921 bytes} [Wed Nov 2 07:29:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:29:58.023550 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dce242-7589-4ef4-8c60-a3e378cad26c None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:58.024021 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dce242-7589-4ef4-8c60-a3e378cad26c None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:58.024315 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dce242-7589-4ef4-8c60-a3e378cad26c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:58.063929 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-06dce242-7589-4ef4-8c60-a3e378cad26c None None] MFA Rules not processed for user `69e48cba3f804cf08b2477f01ae95179`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:29:58.065366 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-06dce242-7589-4ef4-8c60-a3e378cad26c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:58.087988 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:29:58.166385 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 813/1603] 10.222.0.18 () {58 vars in 1018 bytes} [Wed Nov 2 07:29:58 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:29:58.255691 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ef8107-3c1e-4927-8be8-86ba7757e637 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:58.257489 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-11ef8107-3c1e-4927-8be8-86ba7757e637 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:58.278626 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ef8107-3c1e-4927-8be8-86ba7757e637 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:58.284415 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f2bf85-38ac-4844-8e78-c01f5d37277f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e48cba3f804cf08b2477f01ae95179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:58.285099 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f2bf85-38ac-4844-8e78-c01f5d37277f service nova] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:58.285444 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f2bf85-38ac-4844-8e78-c01f5d37277f service nova] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:58.285780 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f2bf85-38ac-4844-8e78-c01f5d37277f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:58.295578 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f2bf85-38ac-4844-8e78-c01f5d37277f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:58.297405 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f2bf85-38ac-4844-8e78-c01f5d37277f service nova] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:58.310424 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 791/1604] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:29:58 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 07:29:59.815259 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dce242-7589-4ef4-8c60-a3e378cad26c None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:59.820396 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-06dce242-7589-4ef4-8c60-a3e378cad26c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:59.892725 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dce242-7589-4ef4-8c60-a3e378cad26c None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:59.897534 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0551dcd2-e189-4a2e-aa74-d26666251790 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:59.898197 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0551dcd2-e189-4a2e-aa74-d26666251790 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:59.898510 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0551dcd2-e189-4a2e-aa74-d26666251790 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:59.898935 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0551dcd2-e189-4a2e-aa74-d26666251790 admin admin] PATH_INFO: `/v3/users/85197edde6aa453ab55854428c8f5973` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:59.900134 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0551dcd2-e189-4a2e-aa74-d26666251790 admin admin] RBAC: Authorizing `identity:delete_user(user_id=85197edde6aa453ab55854428c8f5973)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:59.902736 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0551dcd2-e189-4a2e-aa74-d26666251790 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:59.968161 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-0551dcd2-e189-4a2e-aa74-d26666251790 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '85197edde6aa453ab55854428c8f5973'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:59.979347 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-0551dcd2-e189-4a2e-aa74-d26666251790 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '85197edde6aa453ab55854428c8f5973'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:59.980209 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-0551dcd2-e189-4a2e-aa74-d26666251790 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '85197edde6aa453ab55854428c8f5973'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:59.989447 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-0551dcd2-e189-4a2e-aa74-d26666251790 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '85197edde6aa453ab55854428c8f5973'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:59.990153 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-0551dcd2-e189-4a2e-aa74-d26666251790 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '85197edde6aa453ab55854428c8f5973'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:00.013050 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 814/1605] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Nov 2 07:29:59 2022] DELETE /identity/v3/users/85197edde6aa453ab55854428c8f5973 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:30:00.017905 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75789e32-450c-41e7-b247-37a9d49b5338 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:00.018421 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75789e32-450c-41e7-b247-37a9d49b5338 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:00.018504 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75789e32-450c-41e7-b247-37a9d49b5338 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:00.035647 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:30:00.049557 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-75789e32-450c-41e7-b247-37a9d49b5338 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:30:00.051450 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-75789e32-450c-41e7-b247-37a9d49b5338 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:00.124245 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 792/1606] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:30:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:30:00.133904 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0551dcd2-e189-4a2e-aa74-d26666251790 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:00.135456 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-0551dcd2-e189-4a2e-aa74-d26666251790 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:00.189748 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:30:00.235704 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0551dcd2-e189-4a2e-aa74-d26666251790 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:00.246673 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfca980-7382-4826-b4d9-80252c795015 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3ec6ecb85f94ef59762877d9ea33ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:00.247484 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfca980-7382-4826-b4d9-80252c795015 service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:00.247777 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfca980-7382-4826-b4d9-80252c795015 service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:00.248151 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfca980-7382-4826-b4d9-80252c795015 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:00.259402 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfca980-7382-4826-b4d9-80252c795015 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:00.262314 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfca980-7382-4826-b4d9-80252c795015 service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:00.275273 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 815/1607] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:30:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:30:00.875859 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75789e32-450c-41e7-b247-37a9d49b5338 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:00.877447 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-75789e32-450c-41e7-b247-37a9d49b5338 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:00.896837 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75789e32-450c-41e7-b247-37a9d49b5338 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:00.904270 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25cc12e-0dcb-4ae3-9f0f-31902d106155 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:00.904926 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25cc12e-0dcb-4ae3-9f0f-31902d106155 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:00.906261 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25cc12e-0dcb-4ae3-9f0f-31902d106155 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:00.906725 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25cc12e-0dcb-4ae3-9f0f-31902d106155 admin admin] PATH_INFO: `/v3/users/9bcb3c4382bc4720a91b094cfee1dd78` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:00.908329 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25cc12e-0dcb-4ae3-9f0f-31902d106155 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9bcb3c4382bc4720a91b094cfee1dd78)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:00.910835 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25cc12e-0dcb-4ae3-9f0f-31902d106155 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:00.929797 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfca980-7382-4826-b4d9-80252c795015 service neutron] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:00.932543 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-0cfca980-7382-4826-b4d9-80252c795015 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:00.953545 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfca980-7382-4826-b4d9-80252c795015 service neutron] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:00.968502 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8c9614-685f-47d1-ab24-27c2e7745abf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:00.969849 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8c9614-685f-47d1-ab24-27c2e7745abf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:00.971268 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8c9614-685f-47d1-ab24-27c2e7745abf admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:00.971561 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8c9614-685f-47d1-ab24-27c2e7745abf admin admin] PATH_INFO: `/v3/projects/d4ea076280664004aeb1b0d16be1ee75` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:00.973076 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8c9614-685f-47d1-ab24-27c2e7745abf admin admin] RBAC: Authorizing `identity:delete_project(project_id=d4ea076280664004aeb1b0d16be1ee75)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:00.975143 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8c9614-685f-47d1-ab24-27c2e7745abf admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:01.027473 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-c25cc12e-0dcb-4ae3-9f0f-31902d106155 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9bcb3c4382bc4720a91b094cfee1dd78'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:01.037584 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-c25cc12e-0dcb-4ae3-9f0f-31902d106155 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9bcb3c4382bc4720a91b094cfee1dd78'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:01.038184 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-c25cc12e-0dcb-4ae3-9f0f-31902d106155 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9bcb3c4382bc4720a91b094cfee1dd78'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:01.047189 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-c25cc12e-0dcb-4ae3-9f0f-31902d106155 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9bcb3c4382bc4720a91b094cfee1dd78'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:01.061382 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-c25cc12e-0dcb-4ae3-9f0f-31902d106155 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9bcb3c4382bc4720a91b094cfee1dd78'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:01.069306 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 793/1608] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Nov 2 07:30:00 2022] DELETE /identity/v3/users/9bcb3c4382bc4720a91b094cfee1dd78 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:30:01.073941 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-1c8c9614-685f-47d1-ab24-27c2e7745abf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd4ea076280664004aeb1b0d16be1ee75', 'request_id': 'req-1c8c9614-685f-47d1-ab24-27c2e7745abf'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:01.074259 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e700e72f-29fe-430d-a556-a452a3e697a4 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:01.074574 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e700e72f-29fe-430d-a556-a452a3e697a4 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:01.075037 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e700e72f-29fe-430d-a556-a452a3e697a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:01.081965 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-1c8c9614-685f-47d1-ab24-27c2e7745abf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd4ea076280664004aeb1b0d16be1ee75', 'request_id': 'req-1c8c9614-685f-47d1-ab24-27c2e7745abf'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:01.096610 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-1c8c9614-685f-47d1-ab24-27c2e7745abf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd4ea076280664004aeb1b0d16be1ee75', 'request_id': 'req-1c8c9614-685f-47d1-ab24-27c2e7745abf'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:01.098053 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-e700e72f-29fe-430d-a556-a452a3e697a4 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:30:01.099515 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-e700e72f-29fe-430d-a556-a452a3e697a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:01.125566 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-1c8c9614-685f-47d1-ab24-27c2e7745abf admin admin] The token cache is being invalidate because project d4ea076280664004aeb1b0d16be1ee75 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:30:01.126225 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-1c8c9614-685f-47d1-ab24-27c2e7745abf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:01.128753 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-1c8c9614-685f-47d1-ab24-27c2e7745abf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:01.131160 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 816/1609] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Nov 2 07:30:00 2022] DELETE /identity/v3/projects/d4ea076280664004aeb1b0d16be1ee75 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:30:01.137206 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8c9614-685f-47d1-ab24-27c2e7745abf admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:01.139842 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-1c8c9614-685f-47d1-ab24-27c2e7745abf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:01.201177 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 794/1610] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:30:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:30:01.210068 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e700e72f-29fe-430d-a556-a452a3e697a4 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:01.212239 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-e700e72f-29fe-430d-a556-a452a3e697a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:01.264635 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:30:01.269358 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8c9614-685f-47d1-ab24-27c2e7745abf admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:01.279891 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c277acf1-d404-4594-923c-721f09aa212e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:01.281475 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c277acf1-d404-4594-923c-721f09aa212e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:01.282426 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c277acf1-d404-4594-923c-721f09aa212e admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:01.282988 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c277acf1-d404-4594-923c-721f09aa212e admin admin] PATH_INFO: `/v3/users/7ddc29add40446b2a8ff2abe7c77fd5d` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:01.284802 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c277acf1-d404-4594-923c-721f09aa212e admin admin] RBAC: Authorizing `identity:delete_user(user_id=7ddc29add40446b2a8ff2abe7c77fd5d)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:01.286686 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c277acf1-d404-4594-923c-721f09aa212e admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:01.317129 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e700e72f-29fe-430d-a556-a452a3e697a4 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:01.322236 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c24397-d486-403c-84b9-e7ddce270922 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3ec6ecb85f94ef59762877d9ea33ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:01.322959 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c24397-d486-403c-84b9-e7ddce270922 service neutron] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:01.323242 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c24397-d486-403c-84b9-e7ddce270922 service neutron] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:01.324295 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c24397-d486-403c-84b9-e7ddce270922 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:01.335907 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c24397-d486-403c-84b9-e7ddce270922 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:01.337555 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c24397-d486-403c-84b9-e7ddce270922 service neutron] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:01.349937 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 795/1611] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:30:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 07:30:01.357354 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-c277acf1-d404-4594-923c-721f09aa212e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7ddc29add40446b2a8ff2abe7c77fd5d'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:01.366605 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-c277acf1-d404-4594-923c-721f09aa212e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7ddc29add40446b2a8ff2abe7c77fd5d'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:01.367614 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-c277acf1-d404-4594-923c-721f09aa212e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7ddc29add40446b2a8ff2abe7c77fd5d'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:01.375381 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:30:01.378713 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-c277acf1-d404-4594-923c-721f09aa212e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7ddc29add40446b2a8ff2abe7c77fd5d'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:01.379438 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-c277acf1-d404-4594-923c-721f09aa212e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7ddc29add40446b2a8ff2abe7c77fd5d'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:01.401924 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 817/1612] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Nov 2 07:30:01 2022] DELETE /identity/v3/users/7ddc29add40446b2a8ff2abe7c77fd5d => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:30:01.664375 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c24397-d486-403c-84b9-e7ddce270922 service neutron] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:01.666096 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-e6c24397-d486-403c-84b9-e7ddce270922 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:01.695257 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c24397-d486-403c-84b9-e7ddce270922 service neutron] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:01.703337 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c277acf1-d404-4594-923c-721f09aa212e admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:01.704496 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-c277acf1-d404-4594-923c-721f09aa212e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:01.771616 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36e8924-9892-4732-81a5-26f7df51a538 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:01.772343 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36e8924-9892-4732-81a5-26f7df51a538 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:01.772531 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36e8924-9892-4732-81a5-26f7df51a538 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:01.772799 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36e8924-9892-4732-81a5-26f7df51a538 admin admin] PATH_INFO: `/v3/projects/31e763668e014b6eb992984f254722c1` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:01.773930 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36e8924-9892-4732-81a5-26f7df51a538 admin admin] RBAC: Authorizing `identity:delete_project(project_id=31e763668e014b6eb992984f254722c1)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:01.775847 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36e8924-9892-4732-81a5-26f7df51a538 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:01.786187 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c277acf1-d404-4594-923c-721f09aa212e admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:01.792795 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2d0b85-16f1-4ecd-8da9-a2d126faa3e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:01.793610 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2d0b85-16f1-4ecd-8da9-a2d126faa3e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:01.793947 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2d0b85-16f1-4ecd-8da9-a2d126faa3e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:01.794286 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2d0b85-16f1-4ecd-8da9-a2d126faa3e8 admin admin] PATH_INFO: `/v3/projects/1064d39b9ed94a168e5c5392842b1315` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:01.795779 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2d0b85-16f1-4ecd-8da9-a2d126faa3e8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1064d39b9ed94a168e5c5392842b1315)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:01.798052 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2d0b85-16f1-4ecd-8da9-a2d126faa3e8 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:01.834399 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-d36e8924-9892-4732-81a5-26f7df51a538 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '31e763668e014b6eb992984f254722c1', 'request_id': 'req-d36e8924-9892-4732-81a5-26f7df51a538'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:01.843547 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-d36e8924-9892-4732-81a5-26f7df51a538 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '31e763668e014b6eb992984f254722c1', 'request_id': 'req-d36e8924-9892-4732-81a5-26f7df51a538'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:01.852604 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-d36e8924-9892-4732-81a5-26f7df51a538 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '31e763668e014b6eb992984f254722c1', 'request_id': 'req-d36e8924-9892-4732-81a5-26f7df51a538'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:01.866338 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-d36e8924-9892-4732-81a5-26f7df51a538 admin admin] The token cache is being invalidate because project 31e763668e014b6eb992984f254722c1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81501) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:30:01.866743 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-d36e8924-9892-4732-81a5-26f7df51a538 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:01.867901 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-d36e8924-9892-4732-81a5-26f7df51a538 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:01.869462 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 796/1613] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Nov 2 07:30:01 2022] DELETE /identity/v3/projects/31e763668e014b6eb992984f254722c1 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:30:01.873468 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-dd2d0b85-16f1-4ecd-8da9-a2d126faa3e8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1064d39b9ed94a168e5c5392842b1315', 'request_id': 'req-dd2d0b85-16f1-4ecd-8da9-a2d126faa3e8'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:01.878267 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c7acec-2b08-4f03-813b-13f314ccfb9d None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:01.878567 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c7acec-2b08-4f03-813b-13f314ccfb9d None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:01.878958 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c7acec-2b08-4f03-813b-13f314ccfb9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:01.884016 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-dd2d0b85-16f1-4ecd-8da9-a2d126faa3e8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1064d39b9ed94a168e5c5392842b1315', 'request_id': 'req-dd2d0b85-16f1-4ecd-8da9-a2d126faa3e8'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:01.896845 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-dd2d0b85-16f1-4ecd-8da9-a2d126faa3e8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1064d39b9ed94a168e5c5392842b1315', 'request_id': 'req-dd2d0b85-16f1-4ecd-8da9-a2d126faa3e8'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:01.906960 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-03c7acec-2b08-4f03-813b-13f314ccfb9d None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:30:01.908643 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-03c7acec-2b08-4f03-813b-13f314ccfb9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:01.915873 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-dd2d0b85-16f1-4ecd-8da9-a2d126faa3e8 admin admin] The token cache is being invalidate because project 1064d39b9ed94a168e5c5392842b1315 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:30:01.916154 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-dd2d0b85-16f1-4ecd-8da9-a2d126faa3e8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:01.921128 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-dd2d0b85-16f1-4ecd-8da9-a2d126faa3e8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:01.924319 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 818/1614] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Nov 2 07:30:01 2022] DELETE /identity/v3/projects/1064d39b9ed94a168e5c5392842b1315 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 07:30:01.932045 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0407e47f-97c4-403d-98f7-ced18a5f8c9a None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:01.932741 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0407e47f-97c4-403d-98f7-ced18a5f8c9a None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:01.932741 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0407e47f-97c4-403d-98f7-ced18a5f8c9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:01.972357 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-0407e47f-97c4-403d-98f7-ced18a5f8c9a None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:30:01.975172 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-0407e47f-97c4-403d-98f7-ced18a5f8c9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:02.016360 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:30:02.046728 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 797/1615] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:30:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:30:02.048870 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa95cbaf-a39b-4d64-8f44-c11141bf19a0 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:02.049159 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa95cbaf-a39b-4d64-8f44-c11141bf19a0 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:02.049433 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa95cbaf-a39b-4d64-8f44-c11141bf19a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:02.113068 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-aa95cbaf-a39b-4d64-8f44-c11141bf19a0 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:30:02.116836 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-aa95cbaf-a39b-4d64-8f44-c11141bf19a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:02.120002 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 819/1616] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:30:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:30:02.128711 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc063e95-2023-4e3e-abe7-f6e36cf2347a None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:02.129796 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc063e95-2023-4e3e-abe7-f6e36cf2347a None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:02.130756 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc063e95-2023-4e3e-abe7-f6e36cf2347a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:02.140987 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 798/1617] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:30:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:30:02.142818 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa95cbaf-a39b-4d64-8f44-c11141bf19a0 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:02.159951 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa95cbaf-a39b-4d64-8f44-c11141bf19a0 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:02.170305 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78a51f8-2b98-470c-b0df-867d755fbbd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:02.171185 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78a51f8-2b98-470c-b0df-867d755fbbd1 admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:02.171652 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78a51f8-2b98-470c-b0df-867d755fbbd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:02.172274 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78a51f8-2b98-470c-b0df-867d755fbbd1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:02.173427 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78a51f8-2b98-470c-b0df-867d755fbbd1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:30:02.173761 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78a51f8-2b98-470c-b0df-867d755fbbd1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:02.176259 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78a51f8-2b98-470c-b0df-867d755fbbd1 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:02.181432 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-dc063e95-2023-4e3e-abe7-f6e36cf2347a None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:30:02.183832 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-dc063e95-2023-4e3e-abe7-f6e36cf2347a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:02.196273 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 799/1618] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Nov 2 07:30:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:30:02.201468 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78a51f8-2b98-470c-b0df-867d755fbbd1 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:02.211864 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 820/1619] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:30:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:30:02.214036 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc063e95-2023-4e3e-abe7-f6e36cf2347a None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:02.224840 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:30:02.235668 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78a51f8-2b98-470c-b0df-867d755fbbd1 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:02.238573 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc063e95-2023-4e3e-abe7-f6e36cf2347a None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:02.253629 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819009a1-631d-40f8-a97d-375626766314 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:02.254507 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819009a1-631d-40f8-a97d-375626766314 admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:02.254966 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea9a01c-c710-46c0-b7b1-333325641c28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:02.255104 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819009a1-631d-40f8-a97d-375626766314 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:02.255510 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819009a1-631d-40f8-a97d-375626766314 admin admin] PATH_INFO: `/v3/projects` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:02.255877 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea9a01c-c710-46c0-b7b1-333325641c28 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:02.256263 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea9a01c-c710-46c0-b7b1-333325641c28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:02.256604 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819009a1-631d-40f8-a97d-375626766314 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:02.256743 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea9a01c-c710-46c0-b7b1-333325641c28 admin admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:02.257497 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea9a01c-c710-46c0-b7b1-333325641c28 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:30:02.257883 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea9a01c-c710-46c0-b7b1-333325641c28 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:02.259128 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819009a1-631d-40f8-a97d-375626766314 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:02.259735 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea9a01c-c710-46c0-b7b1-333325641c28 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:02.282547 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 821/1620] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Nov 2 07:30:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:30:02.282547 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea9a01c-c710-46c0-b7b1-333325641c28 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:02.282547 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-eea9a01c-c710-46c0-b7b1-333325641c28 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:02.301413 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea9a01c-c710-46c0-b7b1-333325641c28 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:02.320446 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 800/1621] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Nov 2 07:30:02 2022] POST /identity/v3/projects => generated 385 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:30:02.322929 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819009a1-631d-40f8-a97d-375626766314 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:02.327399 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216084c9-1603-4f57-9d50-021a8cc35b15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:02.328053 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216084c9-1603-4f57-9d50-021a8cc35b15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:02.328336 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216084c9-1603-4f57-9d50-021a8cc35b15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:02.328612 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216084c9-1603-4f57-9d50-021a8cc35b15 admin admin] PATH_INFO: `/v3/users/6a49ee6dd9ac42e6b08f3bbfc85310b4` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:02.330032 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216084c9-1603-4f57-9d50-021a8cc35b15 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6a49ee6dd9ac42e6b08f3bbfc85310b4)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:02.335440 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216084c9-1603-4f57-9d50-021a8cc35b15 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:02.478079 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-216084c9-1603-4f57-9d50-021a8cc35b15 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6a49ee6dd9ac42e6b08f3bbfc85310b4'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:02.479609 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819009a1-631d-40f8-a97d-375626766314 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:02.494658 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0076bc6-c009-41e6-b675-e0a40da2987d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:02.495902 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-216084c9-1603-4f57-9d50-021a8cc35b15 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6a49ee6dd9ac42e6b08f3bbfc85310b4'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:02.496431 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0076bc6-c009-41e6-b675-e0a40da2987d admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:02.496539 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-216084c9-1603-4f57-9d50-021a8cc35b15 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6a49ee6dd9ac42e6b08f3bbfc85310b4'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:02.496717 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0076bc6-c009-41e6-b675-e0a40da2987d admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:02.497050 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0076bc6-c009-41e6-b675-e0a40da2987d admin admin] PATH_INFO: `/v3/projects` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:02.498521 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0076bc6-c009-41e6-b675-e0a40da2987d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:02.500670 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0076bc6-c009-41e6-b675-e0a40da2987d admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:02.551132 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-216084c9-1603-4f57-9d50-021a8cc35b15 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6a49ee6dd9ac42e6b08f3bbfc85310b4'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:02.552290 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-216084c9-1603-4f57-9d50-021a8cc35b15 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6a49ee6dd9ac42e6b08f3bbfc85310b4'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:02.561721 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 801/1622] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Nov 2 07:30:02 2022] POST /identity/v3/projects => generated 379 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:30:02.561721 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0076bc6-c009-41e6-b675-e0a40da2987d admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:02.561721 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-e0076bc6-c009-41e6-b675-e0a40da2987d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:02.606971 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 822/1623] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Nov 2 07:30:02 2022] DELETE /identity/v3/users/6a49ee6dd9ac42e6b08f3bbfc85310b4 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:30:02.608296 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216084c9-1603-4f57-9d50-021a8cc35b15 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:02.615349 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-216084c9-1603-4f57-9d50-021a8cc35b15 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:02.668067 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0076bc6-c009-41e6-b675-e0a40da2987d admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:02.746132 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:30:02.774385 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216084c9-1603-4f57-9d50-021a8cc35b15 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:02.779587 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158e65d3-1021-4a44-a2f2-9ec1a4fe1727 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:02.779587 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158e65d3-1021-4a44-a2f2-9ec1a4fe1727 admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:02.779587 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158e65d3-1021-4a44-a2f2-9ec1a4fe1727 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:02.779883 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ceb72e8-e78d-41c7-ad68-d3fc19f85fe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:02.779993 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158e65d3-1021-4a44-a2f2-9ec1a4fe1727 admin admin] PATH_INFO: `/v3/users` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:02.780064 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ceb72e8-e78d-41c7-ad68-d3fc19f85fe2 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:02.780064 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ceb72e8-e78d-41c7-ad68-d3fc19f85fe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:02.780064 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ceb72e8-e78d-41c7-ad68-d3fc19f85fe2 admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:02.780227 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158e65d3-1021-4a44-a2f2-9ec1a4fe1727 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:02.780304 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ceb72e8-e78d-41c7-ad68-d3fc19f85fe2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:02.782531 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ceb72e8-e78d-41c7-ad68-d3fc19f85fe2 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:02.782932 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158e65d3-1021-4a44-a2f2-9ec1a4fe1727 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:02.808308 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 823/1624] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Nov 2 07:30:02 2022] POST /identity/v3/users => generated 290 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:30:02.809485 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 802/1625] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Nov 2 07:30:02 2022] POST /identity/v3/users => generated 293 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:30:02.811162 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509932a5-f0d0-4f6d-9734-5ab4b753ee76 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:02.811765 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509932a5-f0d0-4f6d-9734-5ab4b753ee76 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:02.812185 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509932a5-f0d0-4f6d-9734-5ab4b753ee76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:02.812284 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158e65d3-1021-4a44-a2f2-9ec1a4fe1727 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:02.823168 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158e65d3-1021-4a44-a2f2-9ec1a4fe1727 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:02.828566 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f37e68a-a808-427a-9049-ff27b94a42f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:02.829183 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f37e68a-a808-427a-9049-ff27b94a42f9 admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:02.829613 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f37e68a-a808-427a-9049-ff27b94a42f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:02.829891 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f37e68a-a808-427a-9049-ff27b94a42f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:02.830627 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f37e68a-a808-427a-9049-ff27b94a42f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:02.832240 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f37e68a-a808-427a-9049-ff27b94a42f9 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:02.836288 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-509932a5-f0d0-4f6d-9734-5ab4b753ee76 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:30:02.838151 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-509932a5-f0d0-4f6d-9734-5ab4b753ee76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:02.846679 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 803/1626] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:30:02 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:30:02.846927 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 824/1627] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:30:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:30:02.848106 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f37e68a-a808-427a-9049-ff27b94a42f9 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:02.852020 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509932a5-f0d0-4f6d-9734-5ab4b753ee76 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:02.874220 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f37e68a-a808-427a-9049-ff27b94a42f9 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:02.883423 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509932a5-f0d0-4f6d-9734-5ab4b753ee76 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:02.887303 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98681e45-c70a-4ef2-a9ad-a759df2d046a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:02.888478 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98681e45-c70a-4ef2-a9ad-a759df2d046a admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:02.889422 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98681e45-c70a-4ef2-a9ad-a759df2d046a admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:02.890056 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df0ea25-869c-45a6-9fea-d00e9ae3f97c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:02.890635 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98681e45-c70a-4ef2-a9ad-a759df2d046a admin admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:02.890741 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df0ea25-869c-45a6-9fea-d00e9ae3f97c admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:02.890966 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df0ea25-869c-45a6-9fea-d00e9ae3f97c admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:02.891237 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df0ea25-869c-45a6-9fea-d00e9ae3f97c admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:02.891929 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df0ea25-869c-45a6-9fea-d00e9ae3f97c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:02.892391 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98681e45-c70a-4ef2-a9ad-a759df2d046a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:02.894150 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df0ea25-869c-45a6-9fea-d00e9ae3f97c admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:02.895014 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98681e45-c70a-4ef2-a9ad-a759df2d046a admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:02.906664 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 825/1628] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:30:02 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:30:02.911591 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df0ea25-869c-45a6-9fea-d00e9ae3f97c admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:02.918340 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-3df0ea25-869c-45a6-9fea-d00e9ae3f97c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:02.919103 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 804/1629] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:30:02 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:30:02.920561 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98681e45-c70a-4ef2-a9ad-a759df2d046a admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:02.934625 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98681e45-c70a-4ef2-a9ad-a759df2d046a admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:02.942690 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fb3a20-4d95-45ab-9b73-b7007828ad69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:02.943458 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fb3a20-4d95-45ab-9b73-b7007828ad69 admin admin] REQUEST_METHOD: `PUT` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:02.943864 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fb3a20-4d95-45ab-9b73-b7007828ad69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:02.944206 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fb3a20-4d95-45ab-9b73-b7007828ad69 admin admin] PATH_INFO: `/v3/projects/d4304f64c7134b76b90e8c2e27621633/users/7f451c7bad88477da8d144ac3efafefe/roles/b494fedd00f24f0b84a529e8672a156a` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:02.964230 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fb3a20-4d95-45ab-9b73-b7007828ad69 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d4304f64c7134b76b90e8c2e27621633, user_id=7f451c7bad88477da8d144ac3efafefe, role_id=b494fedd00f24f0b84a529e8672a156a)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:02.967350 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fb3a20-4d95-45ab-9b73-b7007828ad69 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:02.967636 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:30:02.998014 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 805/1630] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Nov 2 07:30:02 2022] PUT /identity/v3/projects/d4304f64c7134b76b90e8c2e27621633/users/7f451c7bad88477da8d144ac3efafefe/roles/b494fedd00f24f0b84a529e8672a156a => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:30:02.998014 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fb3a20-4d95-45ab-9b73-b7007828ad69 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:03.012266 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df0ea25-869c-45a6-9fea-d00e9ae3f97c admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:03.080724 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fb3a20-4d95-45ab-9b73-b7007828ad69 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:03.081257 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1cc230-99ed-4e39-98b8-92285070154b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3ec6ecb85f94ef59762877d9ea33ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:03.081909 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1cc230-99ed-4e39-98b8-92285070154b service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:03.082266 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1cc230-99ed-4e39-98b8-92285070154b service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:03.082653 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1cc230-99ed-4e39-98b8-92285070154b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:03.086726 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc84dfd-e6f5-4bcd-8058-46035720caae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:03.087539 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc84dfd-e6f5-4bcd-8058-46035720caae admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:03.088011 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc84dfd-e6f5-4bcd-8058-46035720caae admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:03.088468 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc84dfd-e6f5-4bcd-8058-46035720caae admin admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:03.090051 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc84dfd-e6f5-4bcd-8058-46035720caae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:03.091862 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc84dfd-e6f5-4bcd-8058-46035720caae admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:03.093042 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1cc230-99ed-4e39-98b8-92285070154b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:03.094503 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1cc230-99ed-4e39-98b8-92285070154b service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:03.104458 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 826/1631] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:30:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:30:03.106377 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558dab7e-2641-4a20-8103-2a1b27e989f5 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:03.106676 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558dab7e-2641-4a20-8103-2a1b27e989f5 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:03.106956 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558dab7e-2641-4a20-8103-2a1b27e989f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:03.123572 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 806/1632] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:30:02 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:30:03.127920 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc84dfd-e6f5-4bcd-8058-46035720caae admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:03.141129 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc84dfd-e6f5-4bcd-8058-46035720caae admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:03.142304 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-558dab7e-2641-4a20-8103-2a1b27e989f5 None None] MFA Rules not processed for user `7f451c7bad88477da8d144ac3efafefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:30:03.143598 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-558dab7e-2641-4a20-8103-2a1b27e989f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:03.146451 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae14c5c-cbb8-4402-be2d-edc51b6a28ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:03.147166 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae14c5c-cbb8-4402-be2d-edc51b6a28ec admin admin] REQUEST_METHOD: `PUT` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:03.147447 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae14c5c-cbb8-4402-be2d-edc51b6a28ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:03.147884 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae14c5c-cbb8-4402-be2d-edc51b6a28ec admin admin] PATH_INFO: `/v3/projects/266bef2a1b5843f6aaff57309ecd6551/users/b128774a38964c0d9104318572433143/roles/b494fedd00f24f0b84a529e8672a156a` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:03.165050 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae14c5c-cbb8-4402-be2d-edc51b6a28ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=266bef2a1b5843f6aaff57309ecd6551, user_id=b128774a38964c0d9104318572433143, role_id=b494fedd00f24f0b84a529e8672a156a)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:03.168420 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae14c5c-cbb8-4402-be2d-edc51b6a28ec admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:03.186659 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 807/1633] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Nov 2 07:30:03 2022] PUT /identity/v3/projects/266bef2a1b5843f6aaff57309ecd6551/users/b128774a38964c0d9104318572433143/roles/b494fedd00f24f0b84a529e8672a156a => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 07:30:03.197403 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2672377a-da34-4191-b48b-63fcabb1e391 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:03.198073 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2672377a-da34-4191-b48b-63fcabb1e391 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:03.198435 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2672377a-da34-4191-b48b-63fcabb1e391 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:03.225295 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 827/1634] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:30:03 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:30:03.230140 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558dab7e-2641-4a20-8103-2a1b27e989f5 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:03.230673 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-2672377a-da34-4191-b48b-63fcabb1e391 None None] MFA Rules not processed for user `b128774a38964c0d9104318572433143`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:30:03.232106 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-2672377a-da34-4191-b48b-63fcabb1e391 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:03.244540 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558dab7e-2641-4a20-8103-2a1b27e989f5 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:03.284726 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:30:03.310420 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 808/1635] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:30:03 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:30:03.314793 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2672377a-da34-4191-b48b-63fcabb1e391 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:03.318677 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ad5006-8cf6-4360-aaa4-1584bb9bdd8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:03.319301 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ad5006-8cf6-4360-aaa4-1584bb9bdd8f admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:03.319584 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ad5006-8cf6-4360-aaa4-1584bb9bdd8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:03.319859 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ad5006-8cf6-4360-aaa4-1584bb9bdd8f admin admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:03.320635 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ad5006-8cf6-4360-aaa4-1584bb9bdd8f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:03.322637 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ad5006-8cf6-4360-aaa4-1584bb9bdd8f admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:03.326571 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2672377a-da34-4191-b48b-63fcabb1e391 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:03.332277 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1391727a-acb0-4350-9034-a59bc5d893d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:03.333328 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1391727a-acb0-4350-9034-a59bc5d893d0 admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:03.333594 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1391727a-acb0-4350-9034-a59bc5d893d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:03.333920 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1391727a-acb0-4350-9034-a59bc5d893d0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:03.335076 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1391727a-acb0-4350-9034-a59bc5d893d0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:03.338149 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1391727a-acb0-4350-9034-a59bc5d893d0 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:03.367565 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 828/1636] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Nov 2 07:30:03 2022] POST /identity/v3/projects => generated 379 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:30:03.374429 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 809/1637] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Nov 2 07:30:03 2022] POST /identity/v3/projects => generated 383 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:30:03.378074 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ad5006-8cf6-4360-aaa4-1584bb9bdd8f admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:03.378223 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1391727a-acb0-4350-9034-a59bc5d893d0 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:03.394838 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:30:03.491469 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1391727a-acb0-4350-9034-a59bc5d893d0 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:03.500401 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8626aec7-6070-449b-a72a-ea455d6cab6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:03.501121 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ad5006-8cf6-4360-aaa4-1584bb9bdd8f admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:03.501303 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8626aec7-6070-449b-a72a-ea455d6cab6d admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:03.501550 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8626aec7-6070-449b-a72a-ea455d6cab6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:03.501863 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8626aec7-6070-449b-a72a-ea455d6cab6d admin admin] PATH_INFO: `/v3/users` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:03.502693 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8626aec7-6070-449b-a72a-ea455d6cab6d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:03.505013 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8626aec7-6070-449b-a72a-ea455d6cab6d admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:03.518207 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5dce075-003f-48c8-b212-385a575d3883 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:03.519184 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dce075-003f-48c8-b212-385a575d3883 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:03.521062 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dce075-003f-48c8-b212-385a575d3883 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:03.521420 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dce075-003f-48c8-b212-385a575d3883 admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:03.522165 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5dce075-003f-48c8-b212-385a575d3883 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:03.532833 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5dce075-003f-48c8-b212-385a575d3883 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:03.562901 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 810/1638] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Nov 2 07:30:03 2022] POST /identity/v3/users => generated 292 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 07:30:03.562901 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8626aec7-6070-449b-a72a-ea455d6cab6d admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:03.568943 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-8626aec7-6070-449b-a72a-ea455d6cab6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:03.576989 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 829/1639] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Nov 2 07:30:03 2022] POST /identity/v3/users => generated 296 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:30:03.581110 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5dce075-003f-48c8-b212-385a575d3883 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:03.597520 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8626aec7-6070-449b-a72a-ea455d6cab6d admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:03.603645 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5dce075-003f-48c8-b212-385a575d3883 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:03.611000 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0ec281-b0fd-4c89-ad33-5ddc00f27fe0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:03.611650 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0ec281-b0fd-4c89-ad33-5ddc00f27fe0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:03.611962 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0ec281-b0fd-4c89-ad33-5ddc00f27fe0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:03.612237 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0ec281-b0fd-4c89-ad33-5ddc00f27fe0 admin admin] PATH_INFO: `/v3/projects/7f0493557f3543cd9cc6986fd46a9fe0` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:03.613426 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0ec281-b0fd-4c89-ad33-5ddc00f27fe0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7f0493557f3543cd9cc6986fd46a9fe0)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:03.616048 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0ec281-b0fd-4c89-ad33-5ddc00f27fe0 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:03.618067 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d4dbb0-58fc-4903-b40c-de2590266276 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:03.618935 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d4dbb0-58fc-4903-b40c-de2590266276 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:03.619353 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d4dbb0-58fc-4903-b40c-de2590266276 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:03.620120 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d4dbb0-58fc-4903-b40c-de2590266276 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:03.621516 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d4dbb0-58fc-4903-b40c-de2590266276 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:03.623782 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d4dbb0-58fc-4903-b40c-de2590266276 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:03.638132 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 830/1640] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:30:03 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:30:03.640045 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d4dbb0-58fc-4903-b40c-de2590266276 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:03.654854 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d4dbb0-58fc-4903-b40c-de2590266276 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:03.664173 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185e7232-eacd-42e5-896d-27e96742c28e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:03.664835 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185e7232-eacd-42e5-896d-27e96742c28e admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:03.665099 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185e7232-eacd-42e5-896d-27e96742c28e admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:03.665413 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185e7232-eacd-42e5-896d-27e96742c28e admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:03.666622 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185e7232-eacd-42e5-896d-27e96742c28e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:03.668990 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185e7232-eacd-42e5-896d-27e96742c28e admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:03.684561 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 831/1641] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:30:03 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:30:03.685624 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185e7232-eacd-42e5-896d-27e96742c28e admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:03.689613 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-ed0ec281-b0fd-4c89-ad33-5ddc00f27fe0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7f0493557f3543cd9cc6986fd46a9fe0', 'request_id': 'req-ed0ec281-b0fd-4c89-ad33-5ddc00f27fe0'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:03.697965 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-ed0ec281-b0fd-4c89-ad33-5ddc00f27fe0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7f0493557f3543cd9cc6986fd46a9fe0', 'request_id': 'req-ed0ec281-b0fd-4c89-ad33-5ddc00f27fe0'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:03.700558 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185e7232-eacd-42e5-896d-27e96742c28e admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:03.712221 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-ed0ec281-b0fd-4c89-ad33-5ddc00f27fe0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7f0493557f3543cd9cc6986fd46a9fe0', 'request_id': 'req-ed0ec281-b0fd-4c89-ad33-5ddc00f27fe0'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:03.725839 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-ed0ec281-b0fd-4c89-ad33-5ddc00f27fe0 admin admin] The token cache is being invalidate because project 7f0493557f3543cd9cc6986fd46a9fe0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81501) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:30:03.726065 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-ed0ec281-b0fd-4c89-ad33-5ddc00f27fe0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:03.726714 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-ed0ec281-b0fd-4c89-ad33-5ddc00f27fe0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:03.728827 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 811/1642] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Nov 2 07:30:03 2022] DELETE /identity/v3/projects/7f0493557f3543cd9cc6986fd46a9fe0 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:30:03.730483 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0ec281-b0fd-4c89-ad33-5ddc00f27fe0 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:03.732160 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-ed0ec281-b0fd-4c89-ad33-5ddc00f27fe0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:03.808109 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:30:03.842511 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7280e50d-d856-4db7-851e-48db2f0d458d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:03.843319 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7280e50d-d856-4db7-851e-48db2f0d458d admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:03.844046 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7280e50d-d856-4db7-851e-48db2f0d458d admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:03.844525 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7280e50d-d856-4db7-851e-48db2f0d458d admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:03.845748 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7280e50d-d856-4db7-851e-48db2f0d458d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:03.853404 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7280e50d-d856-4db7-851e-48db2f0d458d admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:03.863409 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0ec281-b0fd-4c89-ad33-5ddc00f27fe0 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:03.870272 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6248833-f206-4e5d-bb8c-a0deb38aebf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:03.871092 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6248833-f206-4e5d-bb8c-a0deb38aebf0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:03.871467 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6248833-f206-4e5d-bb8c-a0deb38aebf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:03.871959 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6248833-f206-4e5d-bb8c-a0deb38aebf0 admin admin] PATH_INFO: `/v3/projects/6bf61e05ed9b4e8298296b0a93899cf9/users/3beecfce332640b6a1c224d5293b5ae1/roles/c4bf019fc6c44c71a6c984f6d0bab75f` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:03.892043 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 832/1643] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:30:03 2022] GET /identity/v3/roles => generated 1420 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:30:03.894672 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7280e50d-d856-4db7-851e-48db2f0d458d admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:03.897278 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-7280e50d-d856-4db7-851e-48db2f0d458d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:03.918449 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7280e50d-d856-4db7-851e-48db2f0d458d admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:03.921086 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6248833-f206-4e5d-bb8c-a0deb38aebf0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6bf61e05ed9b4e8298296b0a93899cf9, user_id=3beecfce332640b6a1c224d5293b5ae1, role_id=c4bf019fc6c44c71a6c984f6d0bab75f)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:03.923954 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6248833-f206-4e5d-bb8c-a0deb38aebf0 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:03.924929 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319617f4-9cd4-449e-9a1e-62ce7299c04f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:03.925811 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319617f4-9cd4-449e-9a1e-62ce7299c04f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:03.926249 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319617f4-9cd4-449e-9a1e-62ce7299c04f admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:03.926697 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319617f4-9cd4-449e-9a1e-62ce7299c04f admin admin] PATH_INFO: `/v3/users/0eddebbd0f4b4a9497b3771cd1ffa547` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:03.929104 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319617f4-9cd4-449e-9a1e-62ce7299c04f admin admin] RBAC: Authorizing `identity:delete_user(user_id=0eddebbd0f4b4a9497b3771cd1ffa547)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:03.935436 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319617f4-9cd4-449e-9a1e-62ce7299c04f admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:03.955101 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 812/1644] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Nov 2 07:30:03 2022] PUT /identity/v3/projects/6bf61e05ed9b4e8298296b0a93899cf9/users/3beecfce332640b6a1c224d5293b5ae1/roles/c4bf019fc6c44c71a6c984f6d0bab75f => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:30:03.967127 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6248833-f206-4e5d-bb8c-a0deb38aebf0 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:03.975542 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-c6248833-f206-4e5d-bb8c-a0deb38aebf0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:04.056776 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-319617f4-9cd4-449e-9a1e-62ce7299c04f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0eddebbd0f4b4a9497b3771cd1ffa547'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:04.068480 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-319617f4-9cd4-449e-9a1e-62ce7299c04f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0eddebbd0f4b4a9497b3771cd1ffa547'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:04.069458 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-319617f4-9cd4-449e-9a1e-62ce7299c04f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0eddebbd0f4b4a9497b3771cd1ffa547'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:04.080890 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-319617f4-9cd4-449e-9a1e-62ce7299c04f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0eddebbd0f4b4a9497b3771cd1ffa547'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:04.081808 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-319617f4-9cd4-449e-9a1e-62ce7299c04f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0eddebbd0f4b4a9497b3771cd1ffa547'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:04.091789 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:30:04.108040 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 833/1645] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Nov 2 07:30:03 2022] DELETE /identity/v3/users/0eddebbd0f4b4a9497b3771cd1ffa547 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:30:04.109102 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319617f4-9cd4-449e-9a1e-62ce7299c04f admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:04.110501 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-319617f4-9cd4-449e-9a1e-62ce7299c04f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:04.112990 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6248833-f206-4e5d-bb8c-a0deb38aebf0 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:04.195551 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88ff423-f265-46b1-9fb5-0cd3abd3554b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:04.196678 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88ff423-f265-46b1-9fb5-0cd3abd3554b admin admin] REQUEST_METHOD: `PUT` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:04.197058 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88ff423-f265-46b1-9fb5-0cd3abd3554b admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:04.197452 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88ff423-f265-46b1-9fb5-0cd3abd3554b admin admin] PATH_INFO: `/v3/projects/d8cd884ad482458fa89a9b1fe2847a38/users/59ee6c50550b4254831e31a87ad91c9a/roles/b494fedd00f24f0b84a529e8672a156a` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:04.206471 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319617f4-9cd4-449e-9a1e-62ce7299c04f admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:04.213292 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7076d02-77ce-4c24-a47f-ba4ed8e0fb68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:04.213851 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7076d02-77ce-4c24-a47f-ba4ed8e0fb68 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:04.214993 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7076d02-77ce-4c24-a47f-ba4ed8e0fb68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:04.215284 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7076d02-77ce-4c24-a47f-ba4ed8e0fb68 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:04.216031 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7076d02-77ce-4c24-a47f-ba4ed8e0fb68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:04.217622 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7076d02-77ce-4c24-a47f-ba4ed8e0fb68 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:04.227664 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88ff423-f265-46b1-9fb5-0cd3abd3554b admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8cd884ad482458fa89a9b1fe2847a38, user_id=59ee6c50550b4254831e31a87ad91c9a, role_id=b494fedd00f24f0b84a529e8672a156a)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:04.229066 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 834/1646] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:30:04 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:30:04.234547 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88ff423-f265-46b1-9fb5-0cd3abd3554b admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:04.246782 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7076d02-77ce-4c24-a47f-ba4ed8e0fb68 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:04.259984 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7076d02-77ce-4c24-a47f-ba4ed8e0fb68 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:04.285988 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 813/1647] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Nov 2 07:30:03 2022] PUT /identity/v3/projects/d8cd884ad482458fa89a9b1fe2847a38/users/59ee6c50550b4254831e31a87ad91c9a/roles/b494fedd00f24f0b84a529e8672a156a => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:30:04.287614 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c038f0e-1159-43b8-a60e-15c807459d20 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:04.287980 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c038f0e-1159-43b8-a60e-15c807459d20 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:04.288338 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c038f0e-1159-43b8-a60e-15c807459d20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:04.327851 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-5c038f0e-1159-43b8-a60e-15c807459d20 None None] MFA Rules not processed for user `59ee6c50550b4254831e31a87ad91c9a`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:30:04.329409 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-5c038f0e-1159-43b8-a60e-15c807459d20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:04.366804 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c074abf0-ee00-4fd2-900f-e98fab238dd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:04.368102 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c074abf0-ee00-4fd2-900f-e98fab238dd1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:04.368269 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c074abf0-ee00-4fd2-900f-e98fab238dd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:04.368555 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c074abf0-ee00-4fd2-900f-e98fab238dd1 admin admin] PATH_INFO: `/v3/projects/6bf61e05ed9b4e8298296b0a93899cf9/users/3beecfce332640b6a1c224d5293b5ae1/roles/c4bf019fc6c44c71a6c984f6d0bab75f` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:04.371658 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c074abf0-ee00-4fd2-900f-e98fab238dd1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6bf61e05ed9b4e8298296b0a93899cf9, user_id=3beecfce332640b6a1c224d5293b5ae1, role_id=c4bf019fc6c44c71a6c984f6d0bab75f)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:04.374592 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c074abf0-ee00-4fd2-900f-e98fab238dd1 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:04.387052 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 835/1648] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Nov 2 07:30:04 2022] PUT /identity/v3/projects/6bf61e05ed9b4e8298296b0a93899cf9/users/3beecfce332640b6a1c224d5293b5ae1/roles/c4bf019fc6c44c71a6c984f6d0bab75f => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 07:30:04.399433 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56233c32-89e4-4ef5-9846-39fdd67116e4 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:04.399843 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56233c32-89e4-4ef5-9846-39fdd67116e4 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:04.400194 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56233c32-89e4-4ef5-9846-39fdd67116e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:04.401731 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:30:04.441435 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 814/1649] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:30:04 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:30:04.447157 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c038f0e-1159-43b8-a60e-15c807459d20 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:04.449549 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-56233c32-89e4-4ef5-9846-39fdd67116e4 None None] MFA Rules not processed for user `3beecfce332640b6a1c224d5293b5ae1`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:30:04.451192 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-5c038f0e-1159-43b8-a60e-15c807459d20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:04.452900 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-56233c32-89e4-4ef5-9846-39fdd67116e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:04.479232 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c038f0e-1159-43b8-a60e-15c807459d20 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:04.546661 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:30:04.566329 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f432dad0-0c94-4f01-abf5-12a6b39d147d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:04.567072 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f432dad0-0c94-4f01-abf5-12a6b39d147d admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:04.567420 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f432dad0-0c94-4f01-abf5-12a6b39d147d admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:04.567748 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f432dad0-0c94-4f01-abf5-12a6b39d147d admin admin] PATH_INFO: `/v3/projects` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:04.568449 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f432dad0-0c94-4f01-abf5-12a6b39d147d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:04.569696 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 836/1650] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:30:04 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:30:04.570667 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56233c32-89e4-4ef5-9846-39fdd67116e4 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:04.570773 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f432dad0-0c94-4f01-abf5-12a6b39d147d admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:04.572005 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-56233c32-89e4-4ef5-9846-39fdd67116e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:04.591475 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56233c32-89e4-4ef5-9846-39fdd67116e4 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:04.596401 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7e225c-fabb-456d-bd33-81a836f5d4ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:04.597209 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7e225c-fabb-456d-bd33-81a836f5d4ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:04.597468 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7e225c-fabb-456d-bd33-81a836f5d4ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:04.597962 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7e225c-fabb-456d-bd33-81a836f5d4ea admin admin] PATH_INFO: `/v3/projects/a801965f233d41b58b9b3409c28a57ab` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:04.599243 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7e225c-fabb-456d-bd33-81a836f5d4ea admin admin] RBAC: Authorizing `identity:delete_project(project_id=a801965f233d41b58b9b3409c28a57ab)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:04.601406 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7e225c-fabb-456d-bd33-81a836f5d4ea admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:04.611228 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 815/1651] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Nov 2 07:30:04 2022] POST /identity/v3/projects => generated 383 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:30:04.612545 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f432dad0-0c94-4f01-abf5-12a6b39d147d admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:04.615573 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-f432dad0-0c94-4f01-abf5-12a6b39d147d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:04.684890 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-aa7e225c-fabb-456d-bd33-81a836f5d4ea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a801965f233d41b58b9b3409c28a57ab', 'request_id': 'req-aa7e225c-fabb-456d-bd33-81a836f5d4ea'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:04.707469 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-aa7e225c-fabb-456d-bd33-81a836f5d4ea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a801965f233d41b58b9b3409c28a57ab', 'request_id': 'req-aa7e225c-fabb-456d-bd33-81a836f5d4ea'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:04.715059 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-aa7e225c-fabb-456d-bd33-81a836f5d4ea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a801965f233d41b58b9b3409c28a57ab', 'request_id': 'req-aa7e225c-fabb-456d-bd33-81a836f5d4ea'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:04.729758 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f432dad0-0c94-4f01-abf5-12a6b39d147d admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:04.733502 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-aa7e225c-fabb-456d-bd33-81a836f5d4ea admin admin] The token cache is being invalidate because project a801965f233d41b58b9b3409c28a57ab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:30:04.733822 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-aa7e225c-fabb-456d-bd33-81a836f5d4ea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:04.734593 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-aa7e225c-fabb-456d-bd33-81a836f5d4ea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:04.736461 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 837/1652] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Nov 2 07:30:04 2022] DELETE /identity/v3/projects/a801965f233d41b58b9b3409c28a57ab => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:30:04.740229 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7e225c-fabb-456d-bd33-81a836f5d4ea admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:04.741674 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-aa7e225c-fabb-456d-bd33-81a836f5d4ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:04.783290 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13bfc3c-0231-49ee-99a9-9ee710a5ed6e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43bb326fb82e4d2c83fb2eaf91860dc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:04.783899 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13bfc3c-0231-49ee-99a9-9ee710a5ed6e service cinder] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:04.784179 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13bfc3c-0231-49ee-99a9-9ee710a5ed6e service cinder] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:04.784452 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13bfc3c-0231-49ee-99a9-9ee710a5ed6e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:04.786206 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-c13bfc3c-0231-49ee-99a9-9ee710a5ed6e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:04.811193 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7e225c-fabb-456d-bd33-81a836f5d4ea admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:04.815754 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d38c35-0805-45cf-9335-52fc98da0441 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:04.816352 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d38c35-0805-45cf-9335-52fc98da0441 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:04.816628 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d38c35-0805-45cf-9335-52fc98da0441 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:04.816895 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d38c35-0805-45cf-9335-52fc98da0441 admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:04.817525 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d38c35-0805-45cf-9335-52fc98da0441 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:04.819611 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d38c35-0805-45cf-9335-52fc98da0441 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:04.825671 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:30:04.842443 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 838/1653] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Nov 2 07:30:04 2022] POST /identity/v3/users => generated 298 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:30:04.843681 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b158b1c-d26a-4fee-af4f-9a59b0e4b65b None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:04.843952 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b158b1c-d26a-4fee-af4f-9a59b0e4b65b None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:04.844223 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b158b1c-d26a-4fee-af4f-9a59b0e4b65b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:04.852839 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13bfc3c-0231-49ee-99a9-9ee710a5ed6e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:04.854284 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13bfc3c-0231-49ee-99a9-9ee710a5ed6e service cinder] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:04.863729 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-8b158b1c-d26a-4fee-af4f-9a59b0e4b65b None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:30:04.864178 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 816/1654] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:30:04 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 252 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:30:04.864831 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13bfc3c-0231-49ee-99a9-9ee710a5ed6e service cinder] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:04.864999 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-8b158b1c-d26a-4fee-af4f-9a59b0e4b65b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:04.880452 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 839/1655] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:30:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:30:04.881278 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13bfc3c-0231-49ee-99a9-9ee710a5ed6e service cinder] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:04.881278 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbf72c6-e52c-46cf-aa1e-00ec37feb286 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:04.881940 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbf72c6-e52c-46cf-aa1e-00ec37feb286 admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:04.882283 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbf72c6-e52c-46cf-aa1e-00ec37feb286 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:04.882405 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3739c3c-92d0-4d67-bd27-db5d9f37bfff None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:04.882810 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbf72c6-e52c-46cf-aa1e-00ec37feb286 admin admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:04.882916 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3739c3c-92d0-4d67-bd27-db5d9f37bfff None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:04.883200 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3739c3c-92d0-4d67-bd27-db5d9f37bfff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:04.883634 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbf72c6-e52c-46cf-aa1e-00ec37feb286 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:04.885389 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbf72c6-e52c-46cf-aa1e-00ec37feb286 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:04.901159 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 817/1656] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:30:04 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:30:04.902427 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbf72c6-e52c-46cf-aa1e-00ec37feb286 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:04.913564 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-b3739c3c-92d0-4d67-bd27-db5d9f37bfff None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:30:04.915141 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-b3739c3c-92d0-4d67-bd27-db5d9f37bfff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:04.916863 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbf72c6-e52c-46cf-aa1e-00ec37feb286 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:04.922710 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da9b174-ae77-4c55-b7b9-095ae5ab34cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:04.923382 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da9b174-ae77-4c55-b7b9-095ae5ab34cb admin admin] REQUEST_METHOD: `PUT` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:04.923663 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da9b174-ae77-4c55-b7b9-095ae5ab34cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:04.924025 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da9b174-ae77-4c55-b7b9-095ae5ab34cb admin admin] PATH_INFO: `/v3/projects/d6094e5ca08540afbe75fa5e0fef340a/users/eb2cec8baf6a482eac2e1d5422f18377/roles/c4bf019fc6c44c71a6c984f6d0bab75f` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:04.930413 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 840/1657] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:30:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:30:04.932119 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3739c3c-92d0-4d67-bd27-db5d9f37bfff None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:04.946536 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da9b174-ae77-4c55-b7b9-095ae5ab34cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=d6094e5ca08540afbe75fa5e0fef340a, user_id=eb2cec8baf6a482eac2e1d5422f18377, role_id=c4bf019fc6c44c71a6c984f6d0bab75f)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:04.950008 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3739c3c-92d0-4d67-bd27-db5d9f37bfff None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:04.950131 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da9b174-ae77-4c55-b7b9-095ae5ab34cb admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:04.960159 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1912a838-aa47-4734-afb4-36074cd4cea0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:04.960159 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1912a838-aa47-4734-afb4-36074cd4cea0 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:04.960159 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1912a838-aa47-4734-afb4-36074cd4cea0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:04.960159 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1912a838-aa47-4734-afb4-36074cd4cea0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:04.960594 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1912a838-aa47-4734-afb4-36074cd4cea0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:30:04.960594 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1912a838-aa47-4734-afb4-36074cd4cea0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:04.961812 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1912a838-aa47-4734-afb4-36074cd4cea0 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:04.968636 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 818/1658] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Nov 2 07:30:04 2022] PUT /identity/v3/projects/d6094e5ca08540afbe75fa5e0fef340a/users/eb2cec8baf6a482eac2e1d5422f18377/roles/c4bf019fc6c44c71a6c984f6d0bab75f => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:30:04.972456 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da9b174-ae77-4c55-b7b9-095ae5ab34cb admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:04.983401 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 841/1659] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Nov 2 07:30:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:30:04.987952 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1912a838-aa47-4734-afb4-36074cd4cea0 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:05.000855 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1912a838-aa47-4734-afb4-36074cd4cea0 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:05.108706 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:30:05.118033 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da9b174-ae77-4c55-b7b9-095ae5ab34cb admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:05.134000 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227dea79-8975-466d-8aaf-59fb2b9de05e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:05.137277 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227dea79-8975-466d-8aaf-59fb2b9de05e admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:05.137277 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227dea79-8975-466d-8aaf-59fb2b9de05e admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:05.137277 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227dea79-8975-466d-8aaf-59fb2b9de05e admin admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:05.137277 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227dea79-8975-466d-8aaf-59fb2b9de05e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:05.139399 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227dea79-8975-466d-8aaf-59fb2b9de05e admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:05.145362 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2bfbc3-74e4-4280-9318-3fabeb546ced admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:05.146137 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2bfbc3-74e4-4280-9318-3fabeb546ced admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:05.146448 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2bfbc3-74e4-4280-9318-3fabeb546ced admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:05.147813 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2bfbc3-74e4-4280-9318-3fabeb546ced admin admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:05.148667 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2bfbc3-74e4-4280-9318-3fabeb546ced admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:05.151275 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2bfbc3-74e4-4280-9318-3fabeb546ced admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:05.161906 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 819/1660] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:30:04 2022] GET /identity/v3/roles => generated 1420 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 07:30:05.167680 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227dea79-8975-466d-8aaf-59fb2b9de05e admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:05.186424 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227dea79-8975-466d-8aaf-59fb2b9de05e admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:05.194337 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a505b3af-2624-40bc-b460-6230a5a97b8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:05.195204 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a505b3af-2624-40bc-b460-6230a5a97b8b admin admin] REQUEST_METHOD: `PUT` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:05.195481 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a505b3af-2624-40bc-b460-6230a5a97b8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:05.197860 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a505b3af-2624-40bc-b460-6230a5a97b8b admin admin] PATH_INFO: `/v3/projects/d6094e5ca08540afbe75fa5e0fef340a/users/eb2cec8baf6a482eac2e1d5422f18377/roles/c4bf019fc6c44c71a6c984f6d0bab75f` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:05.202825 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a505b3af-2624-40bc-b460-6230a5a97b8b admin admin] RBAC: Authorizing `identity:create_grant(project_id=d6094e5ca08540afbe75fa5e0fef340a, user_id=eb2cec8baf6a482eac2e1d5422f18377, role_id=c4bf019fc6c44c71a6c984f6d0bab75f)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:05.208866 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a505b3af-2624-40bc-b460-6230a5a97b8b admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:05.213468 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 842/1661] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Nov 2 07:30:04 2022] POST /identity/v3/projects => generated 383 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:30:05.217476 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2bfbc3-74e4-4280-9318-3fabeb546ced admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:05.233151 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 820/1662] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Nov 2 07:30:05 2022] PUT /identity/v3/projects/d6094e5ca08540afbe75fa5e0fef340a/users/eb2cec8baf6a482eac2e1d5422f18377/roles/c4bf019fc6c44c71a6c984f6d0bab75f => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:30:05.244931 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f37d72f-89b5-48af-a72c-8aec4e3a5116 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:05.245920 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f37d72f-89b5-48af-a72c-8aec4e3a5116 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:05.246404 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f37d72f-89b5-48af-a72c-8aec4e3a5116 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:05.322158 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-6f37d72f-89b5-48af-a72c-8aec4e3a5116 None None] MFA Rules not processed for user `eb2cec8baf6a482eac2e1d5422f18377`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:30:05.323618 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-6f37d72f-89b5-48af-a72c-8aec4e3a5116 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:05.401100 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2bfbc3-74e4-4280-9318-3fabeb546ced admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:05.468885 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 821/1663] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:30:05 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:30:05.485618 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f37d72f-89b5-48af-a72c-8aec4e3a5116 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:05.487069 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-6f37d72f-89b5-48af-a72c-8aec4e3a5116 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:05.501555 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:30:05.502099 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac3a7cd-7b5a-47d0-9e77-d15264c2494a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:05.502999 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac3a7cd-7b5a-47d0-9e77-d15264c2494a admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:05.503608 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac3a7cd-7b5a-47d0-9e77-d15264c2494a admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:05.503793 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac3a7cd-7b5a-47d0-9e77-d15264c2494a admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:05.505140 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac3a7cd-7b5a-47d0-9e77-d15264c2494a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:05.507384 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac3a7cd-7b5a-47d0-9e77-d15264c2494a admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:05.537987 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 843/1664] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Nov 2 07:30:05 2022] POST /identity/v3/users => generated 292 bytes in 321 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:30:05.544629 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac3a7cd-7b5a-47d0-9e77-d15264c2494a admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:05.571487 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac3a7cd-7b5a-47d0-9e77-d15264c2494a admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:05.579669 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f075ee4-8b0c-4baa-81d9-1b700935c098 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:05.580240 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f075ee4-8b0c-4baa-81d9-1b700935c098 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:05.580584 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f075ee4-8b0c-4baa-81d9-1b700935c098 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:05.580916 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f075ee4-8b0c-4baa-81d9-1b700935c098 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:05.581633 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f075ee4-8b0c-4baa-81d9-1b700935c098 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:05.583870 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f075ee4-8b0c-4baa-81d9-1b700935c098 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:05.596620 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 844/1665] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:30:05 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 07:30:05.600235 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f075ee4-8b0c-4baa-81d9-1b700935c098 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:05.605761 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f37d72f-89b5-48af-a72c-8aec4e3a5116 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:05.610719 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04cc2f9-b237-4792-9138-c7358e4de0e4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43bb326fb82e4d2c83fb2eaf91860dc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:05.611487 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04cc2f9-b237-4792-9138-c7358e4de0e4 service cinder] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:05.612353 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f075ee4-8b0c-4baa-81d9-1b700935c098 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:05.612825 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04cc2f9-b237-4792-9138-c7358e4de0e4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:05.613237 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04cc2f9-b237-4792-9138-c7358e4de0e4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:05.615962 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-a04cc2f9-b237-4792-9138-c7358e4de0e4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:05.619118 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c56c97f-83bb-43bf-9b21-dd9157108e53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:05.619771 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c56c97f-83bb-43bf-9b21-dd9157108e53 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:05.620666 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c56c97f-83bb-43bf-9b21-dd9157108e53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:05.621425 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c56c97f-83bb-43bf-9b21-dd9157108e53 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:05.622293 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c56c97f-83bb-43bf-9b21-dd9157108e53 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:05.625719 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c56c97f-83bb-43bf-9b21-dd9157108e53 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:05.646120 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 845/1666] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:30:05 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 07:30:05.647175 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c56c97f-83bb-43bf-9b21-dd9157108e53 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:05.663925 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c56c97f-83bb-43bf-9b21-dd9157108e53 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:05.669612 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f0d23e-b98f-48e2-aa46-7571c80d6f8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:05.670329 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f0d23e-b98f-48e2-aa46-7571c80d6f8e admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:05.670796 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f0d23e-b98f-48e2-aa46-7571c80d6f8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:05.670890 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f0d23e-b98f-48e2-aa46-7571c80d6f8e admin admin] PATH_INFO: `/v3/projects/68cdf5ead9864c3496071d8a937dc04b/users/5e40fa9a52e24d148b9a8c4090ba29c4/roles/b494fedd00f24f0b84a529e8672a156a` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:05.687364 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f0d23e-b98f-48e2-aa46-7571c80d6f8e admin admin] RBAC: Authorizing `identity:create_grant(project_id=68cdf5ead9864c3496071d8a937dc04b, user_id=5e40fa9a52e24d148b9a8c4090ba29c4, role_id=b494fedd00f24f0b84a529e8672a156a)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:05.690522 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f0d23e-b98f-48e2-aa46-7571c80d6f8e admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:05.697998 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04cc2f9-b237-4792-9138-c7358e4de0e4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:05.700014 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04cc2f9-b237-4792-9138-c7358e4de0e4 service cinder] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:05.708599 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 846/1667] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Nov 2 07:30:05 2022] PUT /identity/v3/projects/68cdf5ead9864c3496071d8a937dc04b/users/5e40fa9a52e24d148b9a8c4090ba29c4/roles/b494fedd00f24f0b84a529e8672a156a => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 07:30:05.710725 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 822/1668] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:30:05 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:30:05.711304 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f0d23e-b98f-48e2-aa46-7571c80d6f8e admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:05.712921 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-90f0d23e-b98f-48e2-aa46-7571c80d6f8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:05.721401 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1236d0b-1911-4633-abf3-ed4aee0a8ede None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:05.721691 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1236d0b-1911-4633-abf3-ed4aee0a8ede None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:05.723335 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1236d0b-1911-4633-abf3-ed4aee0a8ede None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:05.768465 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-d1236d0b-1911-4633-abf3-ed4aee0a8ede None None] MFA Rules not processed for user `5e40fa9a52e24d148b9a8c4090ba29c4`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:30:05.770252 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-d1236d0b-1911-4633-abf3-ed4aee0a8ede None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:05.783482 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:30:05.835771 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f0d23e-b98f-48e2-aa46-7571c80d6f8e admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:05.840903 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10f2740-7dd4-4172-b1ac-040515fc6368 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:05.841538 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10f2740-7dd4-4172-b1ac-040515fc6368 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:05.841946 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10f2740-7dd4-4172-b1ac-040515fc6368 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:05.842223 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10f2740-7dd4-4172-b1ac-040515fc6368 admin admin] PATH_INFO: `/v3/users/7f451c7bad88477da8d144ac3efafefe` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:05.843581 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10f2740-7dd4-4172-b1ac-040515fc6368 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7f451c7bad88477da8d144ac3efafefe)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:05.845658 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10f2740-7dd4-4172-b1ac-040515fc6368 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:05.934265 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-b10f2740-7dd4-4172-b1ac-040515fc6368 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7f451c7bad88477da8d144ac3efafefe'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:05.942229 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 823/1669] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:30:05 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:30:05.949250 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-b10f2740-7dd4-4172-b1ac-040515fc6368 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7f451c7bad88477da8d144ac3efafefe'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:05.950082 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-b10f2740-7dd4-4172-b1ac-040515fc6368 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7f451c7bad88477da8d144ac3efafefe'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:05.959265 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-b10f2740-7dd4-4172-b1ac-040515fc6368 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7f451c7bad88477da8d144ac3efafefe'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:05.960059 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-b10f2740-7dd4-4172-b1ac-040515fc6368 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7f451c7bad88477da8d144ac3efafefe'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:05.986202 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 847/1670] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Nov 2 07:30:05 2022] DELETE /identity/v3/users/7f451c7bad88477da8d144ac3efafefe => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:30:05.988167 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5730bd9b-40e6-44cb-8f79-1ba0c7a11b5d None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:05.988561 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5730bd9b-40e6-44cb-8f79-1ba0c7a11b5d None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:05.988841 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5730bd9b-40e6-44cb-8f79-1ba0c7a11b5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:06.005088 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10f2740-7dd4-4172-b1ac-040515fc6368 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:06.007330 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-b10f2740-7dd4-4172-b1ac-040515fc6368 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:06.026446 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-5730bd9b-40e6-44cb-8f79-1ba0c7a11b5d None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:30:06.027894 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-5730bd9b-40e6-44cb-8f79-1ba0c7a11b5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:06.086486 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:30:06.103369 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 824/1671] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:30:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:30:06.115460 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5730bd9b-40e6-44cb-8f79-1ba0c7a11b5d None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:06.115821 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10f2740-7dd4-4172-b1ac-040515fc6368 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:06.117810 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-5730bd9b-40e6-44cb-8f79-1ba0c7a11b5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:06.123594 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dec8643-cc1c-4ce8-a187-dc6d3370a775 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43bb326fb82e4d2c83fb2eaf91860dc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:06.124433 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dec8643-cc1c-4ce8-a187-dc6d3370a775 service cinder] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:06.125005 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dec8643-cc1c-4ce8-a187-dc6d3370a775 service cinder] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:06.125451 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dec8643-cc1c-4ce8-a187-dc6d3370a775 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:06.129194 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-2dec8643-cc1c-4ce8-a187-dc6d3370a775 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:06.196472 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5730bd9b-40e6-44cb-8f79-1ba0c7a11b5d None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:06.197205 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:30:06.201946 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dec8643-cc1c-4ce8-a187-dc6d3370a775 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:06.202873 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b635069-78e8-416a-9ce1-20873f9ed61f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3ec6ecb85f94ef59762877d9ea33ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:06.203809 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b635069-78e8-416a-9ce1-20873f9ed61f service neutron] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:06.204133 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dec8643-cc1c-4ce8-a187-dc6d3370a775 service cinder] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:06.204579 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b635069-78e8-416a-9ce1-20873f9ed61f service neutron] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:06.205037 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b635069-78e8-416a-9ce1-20873f9ed61f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:06.215593 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b635069-78e8-416a-9ce1-20873f9ed61f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:06.216106 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 848/1672] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:30:06 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 07:30:06.217071 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b635069-78e8-416a-9ce1-20873f9ed61f service neutron] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:06.227072 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 825/1673] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:30:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 07:30:06.237007 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dec8643-cc1c-4ce8-a187-dc6d3370a775 service cinder] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:06.238446 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-2dec8643-cc1c-4ce8-a187-dc6d3370a775 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:06.252552 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dec8643-cc1c-4ce8-a187-dc6d3370a775 service cinder] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:06.257091 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f10699-2c48-4057-adf8-443b439edcf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:06.257715 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f10699-2c48-4057-adf8-443b439edcf8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:06.258001 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f10699-2c48-4057-adf8-443b439edcf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:06.258284 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f10699-2c48-4057-adf8-443b439edcf8 admin admin] PATH_INFO: `/v3/users/5e40fa9a52e24d148b9a8c4090ba29c4` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:06.259448 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f10699-2c48-4057-adf8-443b439edcf8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5e40fa9a52e24d148b9a8c4090ba29c4)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:06.261414 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f10699-2c48-4057-adf8-443b439edcf8 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:06.315633 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-12f10699-2c48-4057-adf8-443b439edcf8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5e40fa9a52e24d148b9a8c4090ba29c4'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:06.324485 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-12f10699-2c48-4057-adf8-443b439edcf8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5e40fa9a52e24d148b9a8c4090ba29c4'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:06.325205 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-12f10699-2c48-4057-adf8-443b439edcf8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5e40fa9a52e24d148b9a8c4090ba29c4'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:06.333069 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-12f10699-2c48-4057-adf8-443b439edcf8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5e40fa9a52e24d148b9a8c4090ba29c4'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:06.333787 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-12f10699-2c48-4057-adf8-443b439edcf8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5e40fa9a52e24d148b9a8c4090ba29c4'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:06.351920 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 849/1674] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Nov 2 07:30:06 2022] DELETE /identity/v3/users/5e40fa9a52e24d148b9a8c4090ba29c4 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:30:06.357136 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed41d05-4cbb-46fe-8e00-646410b81f48 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:06.357436 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed41d05-4cbb-46fe-8e00-646410b81f48 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:06.357744 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed41d05-4cbb-46fe-8e00-646410b81f48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:06.381616 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-2ed41d05-4cbb-46fe-8e00-646410b81f48 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:30:06.383412 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-2ed41d05-4cbb-46fe-8e00-646410b81f48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:06.466100 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 826/1675] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:30:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:30:06.477075 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f10699-2c48-4057-adf8-443b439edcf8 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:06.478352 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-12f10699-2c48-4057-adf8-443b439edcf8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:06.584943 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f10699-2c48-4057-adf8-443b439edcf8 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:06.590702 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41a9c94-413f-4ad6-899f-cb43eca1f012 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3ec6ecb85f94ef59762877d9ea33ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:06.593306 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41a9c94-413f-4ad6-899f-cb43eca1f012 service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:06.594042 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41a9c94-413f-4ad6-899f-cb43eca1f012 service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:06.594590 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41a9c94-413f-4ad6-899f-cb43eca1f012 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:06.613476 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41a9c94-413f-4ad6-899f-cb43eca1f012 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:06.616165 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41a9c94-413f-4ad6-899f-cb43eca1f012 service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:06.635655 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 850/1676] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:30:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:30:06.785741 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed41d05-4cbb-46fe-8e00-646410b81f48 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:06.787065 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-2ed41d05-4cbb-46fe-8e00-646410b81f48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:06.812729 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed41d05-4cbb-46fe-8e00-646410b81f48 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:06.819506 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfc8477-44dd-4085-8efc-a46d075316a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:06.820600 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfc8477-44dd-4085-8efc-a46d075316a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:06.821209 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfc8477-44dd-4085-8efc-a46d075316a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:06.821738 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfc8477-44dd-4085-8efc-a46d075316a6 admin admin] PATH_INFO: `/v3/projects/d4304f64c7134b76b90e8c2e27621633` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:06.824598 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfc8477-44dd-4085-8efc-a46d075316a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d4304f64c7134b76b90e8c2e27621633)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:06.828208 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfc8477-44dd-4085-8efc-a46d075316a6 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:06.924082 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-4dfc8477-44dd-4085-8efc-a46d075316a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd4304f64c7134b76b90e8c2e27621633', 'request_id': 'req-4dfc8477-44dd-4085-8efc-a46d075316a6'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:06.930631 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-4dfc8477-44dd-4085-8efc-a46d075316a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd4304f64c7134b76b90e8c2e27621633', 'request_id': 'req-4dfc8477-44dd-4085-8efc-a46d075316a6'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:06.942765 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-4dfc8477-44dd-4085-8efc-a46d075316a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd4304f64c7134b76b90e8c2e27621633', 'request_id': 'req-4dfc8477-44dd-4085-8efc-a46d075316a6'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:06.964605 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-4dfc8477-44dd-4085-8efc-a46d075316a6 admin admin] The token cache is being invalidate because project d4304f64c7134b76b90e8c2e27621633 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81501) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:30:06.964900 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-4dfc8477-44dd-4085-8efc-a46d075316a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:06.965695 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-4dfc8477-44dd-4085-8efc-a46d075316a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:06.971024 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 827/1677] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Nov 2 07:30:06 2022] DELETE /identity/v3/projects/d4304f64c7134b76b90e8c2e27621633 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:30:06.971742 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41a9c94-413f-4ad6-899f-cb43eca1f012 service neutron] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:06.982843 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41a9c94-413f-4ad6-899f-cb43eca1f012 service neutron] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:07.084407 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c673dbd5-1cd3-41a2-bf02-f25a4783baab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:07.085159 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c673dbd5-1cd3-41a2-bf02-f25a4783baab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:07.085502 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c673dbd5-1cd3-41a2-bf02-f25a4783baab admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:07.085807 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c673dbd5-1cd3-41a2-bf02-f25a4783baab admin admin] PATH_INFO: `/v3/users/3beecfce332640b6a1c224d5293b5ae1` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:07.087396 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c673dbd5-1cd3-41a2-bf02-f25a4783baab admin admin] RBAC: Authorizing `identity:delete_user(user_id=3beecfce332640b6a1c224d5293b5ae1)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:07.089782 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c673dbd5-1cd3-41a2-bf02-f25a4783baab admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:07.100767 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:30:07.109589 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfc8477-44dd-4085-8efc-a46d075316a6 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:07.111369 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-4dfc8477-44dd-4085-8efc-a46d075316a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:07.129070 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfc8477-44dd-4085-8efc-a46d075316a6 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:07.135186 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe0fba0-c1d9-44ad-8e89-0d902606ae95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:07.136751 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe0fba0-c1d9-44ad-8e89-0d902606ae95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:07.137212 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe0fba0-c1d9-44ad-8e89-0d902606ae95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:07.137720 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe0fba0-c1d9-44ad-8e89-0d902606ae95 admin admin] PATH_INFO: `/v3/projects/68cdf5ead9864c3496071d8a937dc04b` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:07.139160 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe0fba0-c1d9-44ad-8e89-0d902606ae95 admin admin] RBAC: Authorizing `identity:delete_project(project_id=68cdf5ead9864c3496071d8a937dc04b)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:07.141376 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe0fba0-c1d9-44ad-8e89-0d902606ae95 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:07.158403 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:30:07.172461 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-c673dbd5-1cd3-41a2-bf02-f25a4783baab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3beecfce332640b6a1c224d5293b5ae1'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:07.185316 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-c673dbd5-1cd3-41a2-bf02-f25a4783baab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3beecfce332640b6a1c224d5293b5ae1'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:07.186186 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-c673dbd5-1cd3-41a2-bf02-f25a4783baab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3beecfce332640b6a1c224d5293b5ae1'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:07.194833 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-c673dbd5-1cd3-41a2-bf02-f25a4783baab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3beecfce332640b6a1c224d5293b5ae1'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:07.195552 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-c673dbd5-1cd3-41a2-bf02-f25a4783baab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3beecfce332640b6a1c224d5293b5ae1'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:07.217067 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-8fe0fba0-c1d9-44ad-8e89-0d902606ae95 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '68cdf5ead9864c3496071d8a937dc04b', 'request_id': 'req-8fe0fba0-c1d9-44ad-8e89-0d902606ae95'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:07.226156 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-8fe0fba0-c1d9-44ad-8e89-0d902606ae95 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '68cdf5ead9864c3496071d8a937dc04b', 'request_id': 'req-8fe0fba0-c1d9-44ad-8e89-0d902606ae95'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:07.233608 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 851/1678] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Nov 2 07:30:06 2022] DELETE /identity/v3/users/3beecfce332640b6a1c224d5293b5ae1 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:30:07.238145 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-8fe0fba0-c1d9-44ad-8e89-0d902606ae95 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '68cdf5ead9864c3496071d8a937dc04b', 'request_id': 'req-8fe0fba0-c1d9-44ad-8e89-0d902606ae95'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:07.266579 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-8fe0fba0-c1d9-44ad-8e89-0d902606ae95 admin admin] The token cache is being invalidate because project 68cdf5ead9864c3496071d8a937dc04b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81501) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:30:07.267011 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-8fe0fba0-c1d9-44ad-8e89-0d902606ae95 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:07.267788 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-8fe0fba0-c1d9-44ad-8e89-0d902606ae95 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:07.272196 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 828/1679] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Nov 2 07:30:07 2022] DELETE /identity/v3/projects/68cdf5ead9864c3496071d8a937dc04b => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 07:30:07.276270 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c2f667-8e95-4e9a-b144-357079ea794e None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:07.277128 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c2f667-8e95-4e9a-b144-357079ea794e None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:07.277413 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c2f667-8e95-4e9a-b144-357079ea794e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:07.298573 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-70c2f667-8e95-4e9a-b144-357079ea794e None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:30:07.300585 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-70c2f667-8e95-4e9a-b144-357079ea794e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:07.398546 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 852/1680] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:30:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:30:07.399808 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4872bd1a-8ac7-4bbc-9baa-6939df4d93f2 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:07.400613 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4872bd1a-8ac7-4bbc-9baa-6939df4d93f2 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:07.401246 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4872bd1a-8ac7-4bbc-9baa-6939df4d93f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:07.445509 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-4872bd1a-8ac7-4bbc-9baa-6939df4d93f2 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:30:07.446807 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-4872bd1a-8ac7-4bbc-9baa-6939df4d93f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:07.476579 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 829/1681] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:30:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:30:07.487060 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c2f667-8e95-4e9a-b144-357079ea794e None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:07.503424 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c2f667-8e95-4e9a-b144-357079ea794e None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:07.514856 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1592dad-c26e-448f-838f-8d1989bd1ed1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:07.515481 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1592dad-c26e-448f-838f-8d1989bd1ed1 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:07.515758 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1592dad-c26e-448f-838f-8d1989bd1ed1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:07.516028 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1592dad-c26e-448f-838f-8d1989bd1ed1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:07.516994 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1592dad-c26e-448f-838f-8d1989bd1ed1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:30:07.517337 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1592dad-c26e-448f-838f-8d1989bd1ed1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:07.519142 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1592dad-c26e-448f-838f-8d1989bd1ed1 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:07.538457 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 853/1682] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Nov 2 07:30:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:30:07.541376 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4872bd1a-8ac7-4bbc-9baa-6939df4d93f2 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:07.552197 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4872bd1a-8ac7-4bbc-9baa-6939df4d93f2 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:07.557258 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a61aad-1db4-4673-8d8a-32455a1418ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:07.557521 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a61aad-1db4-4673-8d8a-32455a1418ab admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:07.557859 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a61aad-1db4-4673-8d8a-32455a1418ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:07.558134 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a61aad-1db4-4673-8d8a-32455a1418ab admin admin] PATH_INFO: `/v3/projects` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:07.558808 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a61aad-1db4-4673-8d8a-32455a1418ab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:07.560693 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a61aad-1db4-4673-8d8a-32455a1418ab admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:07.579514 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1592dad-c26e-448f-838f-8d1989bd1ed1 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:07.581289 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-c1592dad-c26e-448f-838f-8d1989bd1ed1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:07.593200 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 830/1683] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Nov 2 07:30:07 2022] POST /identity/v3/projects => generated 377 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:30:07.597453 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a61aad-1db4-4673-8d8a-32455a1418ab admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:07.660598 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1592dad-c26e-448f-838f-8d1989bd1ed1 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:07.689413 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a61aad-1db4-4673-8d8a-32455a1418ab admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:07.694400 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e05785d-e6e0-4c0f-8b7e-370250fa1c23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:07.695274 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e05785d-e6e0-4c0f-8b7e-370250fa1c23 admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:07.695629 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e05785d-e6e0-4c0f-8b7e-370250fa1c23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:07.695907 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e05785d-e6e0-4c0f-8b7e-370250fa1c23 admin admin] PATH_INFO: `/v3/users` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:07.696532 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e05785d-e6e0-4c0f-8b7e-370250fa1c23 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:07.698985 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e05785d-e6e0-4c0f-8b7e-370250fa1c23 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:07.709113 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:30:07.732869 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 831/1684] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Nov 2 07:30:07 2022] POST /identity/v3/users => generated 289 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 07:30:07.732869 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e05785d-e6e0-4c0f-8b7e-370250fa1c23 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:07.734994 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-1e05785d-e6e0-4c0f-8b7e-370250fa1c23 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:07.740718 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c948d116-9a4a-45e0-9ab8-c43f539c4890 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43bb326fb82e4d2c83fb2eaf91860dc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:07.741393 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c948d116-9a4a-45e0-9ab8-c43f539c4890 service cinder] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:07.741744 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c948d116-9a4a-45e0-9ab8-c43f539c4890 service cinder] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:07.742094 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c948d116-9a4a-45e0-9ab8-c43f539c4890 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:07.744399 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-c948d116-9a4a-45e0-9ab8-c43f539c4890 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:07.750759 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e05785d-e6e0-4c0f-8b7e-370250fa1c23 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:07.756521 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b3062d-a94c-4fb8-8929-a6762cf04cc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:07.757153 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b3062d-a94c-4fb8-8929-a6762cf04cc5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:07.757435 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b3062d-a94c-4fb8-8929-a6762cf04cc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:07.757710 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b3062d-a94c-4fb8-8929-a6762cf04cc5 admin admin] PATH_INFO: `/v3/projects/6bf61e05ed9b4e8298296b0a93899cf9` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:07.758839 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b3062d-a94c-4fb8-8929-a6762cf04cc5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6bf61e05ed9b4e8298296b0a93899cf9)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:07.760823 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b3062d-a94c-4fb8-8929-a6762cf04cc5 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:07.812488 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c948d116-9a4a-45e0-9ab8-c43f539c4890 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:07.813926 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c948d116-9a4a-45e0-9ab8-c43f539c4890 service cinder] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:07.820142 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-20b3062d-a94c-4fb8-8929-a6762cf04cc5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6bf61e05ed9b4e8298296b0a93899cf9', 'request_id': 'req-20b3062d-a94c-4fb8-8929-a6762cf04cc5'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:07.826019 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 854/1685] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:30:07 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 247 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 07:30:07.829784 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-20b3062d-a94c-4fb8-8929-a6762cf04cc5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6bf61e05ed9b4e8298296b0a93899cf9', 'request_id': 'req-20b3062d-a94c-4fb8-8929-a6762cf04cc5'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:07.832042 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c948d116-9a4a-45e0-9ab8-c43f539c4890 service cinder] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:07.832649 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=83,keystone:DELETE=11,keystone:INSERT=6 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:30:07.842281 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-20b3062d-a94c-4fb8-8929-a6762cf04cc5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6bf61e05ed9b4e8298296b0a93899cf9', 'request_id': 'req-20b3062d-a94c-4fb8-8929-a6762cf04cc5'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:07.846130 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c948d116-9a4a-45e0-9ab8-c43f539c4890 service cinder] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:07.857923 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-20b3062d-a94c-4fb8-8929-a6762cf04cc5 admin admin] The token cache is being invalidate because project 6bf61e05ed9b4e8298296b0a93899cf9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81501) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:30:07.858335 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-20b3062d-a94c-4fb8-8929-a6762cf04cc5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:07.859042 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-20b3062d-a94c-4fb8-8929-a6762cf04cc5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:07.860675 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 832/1686] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Nov 2 07:30:07 2022] DELETE /identity/v3/projects/6bf61e05ed9b4e8298296b0a93899cf9 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:30:07.865559 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0541dd29-71a4-4706-8c10-2520c36a6571 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:07.865753 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0541dd29-71a4-4706-8c10-2520c36a6571 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:07.865989 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0541dd29-71a4-4706-8c10-2520c36a6571 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:07.920211 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-0541dd29-71a4-4706-8c10-2520c36a6571 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:30:07.921688 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-0541dd29-71a4-4706-8c10-2520c36a6571 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:07.935768 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25195c29-6b58-402a-b3d1-9e627c437644 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:07.936876 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25195c29-6b58-402a-b3d1-9e627c437644 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:07.937632 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25195c29-6b58-402a-b3d1-9e627c437644 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:07.938085 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25195c29-6b58-402a-b3d1-9e627c437644 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:07.938817 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25195c29-6b58-402a-b3d1-9e627c437644 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:07.942464 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25195c29-6b58-402a-b3d1-9e627c437644 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:07.963086 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 855/1687] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:30:07 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:30:07.965633 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25195c29-6b58-402a-b3d1-9e627c437644 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:07.967790 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-25195c29-6b58-402a-b3d1-9e627c437644 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:07.985374 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25195c29-6b58-402a-b3d1-9e627c437644 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:07.990588 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e14923f-7f2f-4b87-a71d-e2cb6fc6319d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:07.991317 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e14923f-7f2f-4b87-a71d-e2cb6fc6319d admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:07.991877 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e14923f-7f2f-4b87-a71d-e2cb6fc6319d admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:07.991986 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e14923f-7f2f-4b87-a71d-e2cb6fc6319d admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:07.992827 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e14923f-7f2f-4b87-a71d-e2cb6fc6319d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:07.994655 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e14923f-7f2f-4b87-a71d-e2cb6fc6319d admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:08.007563 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 856/1688] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:30:07 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:30:08.009737 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 833/1689] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:30:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:30:08.016336 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e14923f-7f2f-4b87-a71d-e2cb6fc6319d admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:08.017983 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43352308-17fe-4003-b3ab-3873588fea55 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:08.018477 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43352308-17fe-4003-b3ab-3873588fea55 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:08.018896 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43352308-17fe-4003-b3ab-3873588fea55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:08.037539 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e14923f-7f2f-4b87-a71d-e2cb6fc6319d admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:08.046705 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-43352308-17fe-4003-b3ab-3873588fea55 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:30:08.047353 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a9df7e-132e-41ee-ad65-7596e781b9db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:08.048801 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a9df7e-132e-41ee-ad65-7596e781b9db admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:08.049406 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a9df7e-132e-41ee-ad65-7596e781b9db admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:08.049958 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a9df7e-132e-41ee-ad65-7596e781b9db admin admin] PATH_INFO: `/v3/projects/e9d315fca9c14f30afdcc8e791a7bfbe/users/a79c7d7a31074936bc16152715c54dcc/roles/b494fedd00f24f0b84a529e8672a156a` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:08.050546 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-43352308-17fe-4003-b3ab-3873588fea55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:08.072037 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a9df7e-132e-41ee-ad65-7596e781b9db admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9d315fca9c14f30afdcc8e791a7bfbe, user_id=a79c7d7a31074936bc16152715c54dcc, role_id=b494fedd00f24f0b84a529e8672a156a)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:08.073027 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 834/1690] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:30:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:30:08.075474 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a9df7e-132e-41ee-ad65-7596e781b9db admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:08.079329 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43352308-17fe-4003-b3ab-3873588fea55 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:08.098121 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43352308-17fe-4003-b3ab-3873588fea55 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:08.102478 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 857/1691] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Nov 2 07:30:08 2022] PUT /identity/v3/projects/e9d315fca9c14f30afdcc8e791a7bfbe/users/a79c7d7a31074936bc16152715c54dcc/roles/b494fedd00f24f0b84a529e8672a156a => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 07:30:08.113554 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e31738-b4fe-4dc9-9766-c5a1a726dd5e None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:08.114746 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e31738-b4fe-4dc9-9766-c5a1a726dd5e None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:08.115088 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e31738-b4fe-4dc9-9766-c5a1a726dd5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:08.168332 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-49e31738-b4fe-4dc9-9766-c5a1a726dd5e None None] MFA Rules not processed for user `a79c7d7a31074936bc16152715c54dcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:30:08.170002 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-49e31738-b4fe-4dc9-9766-c5a1a726dd5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:08.205339 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f99a63c-a47a-405b-a7ca-e4545b025a78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:08.206051 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f99a63c-a47a-405b-a7ca-e4545b025a78 admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:08.206442 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f99a63c-a47a-405b-a7ca-e4545b025a78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:08.207009 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f99a63c-a47a-405b-a7ca-e4545b025a78 admin admin] PATH_INFO: `/v3/domains` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:08.207580 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f99a63c-a47a-405b-a7ca-e4545b025a78 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:30:08.207888 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f99a63c-a47a-405b-a7ca-e4545b025a78 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:08.209915 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f99a63c-a47a-405b-a7ca-e4545b025a78 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:08.224577 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:30:08.230433 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 835/1692] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Nov 2 07:30:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:30:08.234113 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f99a63c-a47a-405b-a7ca-e4545b025a78 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:08.259488 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f99a63c-a47a-405b-a7ca-e4545b025a78 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:08.268143 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1496c8c0-62e5-462b-b1f4-a7e2e151e0f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:08.269229 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1496c8c0-62e5-462b-b1f4-a7e2e151e0f5 admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:08.270759 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1496c8c0-62e5-462b-b1f4-a7e2e151e0f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:08.271135 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1496c8c0-62e5-462b-b1f4-a7e2e151e0f5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:08.271888 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1496c8c0-62e5-462b-b1f4-a7e2e151e0f5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:08.273914 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1496c8c0-62e5-462b-b1f4-a7e2e151e0f5 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:08.299360 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 858/1693] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:30:08 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:30:08.311858 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e31738-b4fe-4dc9-9766-c5a1a726dd5e None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:08.313294 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-49e31738-b4fe-4dc9-9766-c5a1a726dd5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:08.338394 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 836/1694] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Nov 2 07:30:08 2022] POST /identity/v3/projects => generated 359 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:30:08.340922 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1496c8c0-62e5-462b-b1f4-a7e2e151e0f5 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:08.410322 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:DELETE=3,keystone:SELECT=96,keystone:INSERT=1 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:30:08.436005 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1496c8c0-62e5-462b-b1f4-a7e2e151e0f5 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:08.450934 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3e83cf-0abd-4421-a5f4-d2ae65c9547b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:08.451865 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3e83cf-0abd-4421-a5f4-d2ae65c9547b admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:08.452146 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3e83cf-0abd-4421-a5f4-d2ae65c9547b admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:08.452612 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3e83cf-0abd-4421-a5f4-d2ae65c9547b admin admin] PATH_INFO: `/v3/users` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:08.453622 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3e83cf-0abd-4421-a5f4-d2ae65c9547b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:08.454539 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e31738-b4fe-4dc9-9766-c5a1a726dd5e None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:08.457204 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3e83cf-0abd-4421-a5f4-d2ae65c9547b admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:08.524236 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 837/1695] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Nov 2 07:30:08 2022] POST /identity/v3/users => generated 280 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:30:08.530098 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3e83cf-0abd-4421-a5f4-d2ae65c9547b admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:08.546850 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3e83cf-0abd-4421-a5f4-d2ae65c9547b admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:08.556541 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfc4208-96a5-4ea6-ad45-5c5d89088ba1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:08.557457 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfc4208-96a5-4ea6-ad45-5c5d89088ba1 admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:08.557990 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfc4208-96a5-4ea6-ad45-5c5d89088ba1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:08.558507 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfc4208-96a5-4ea6-ad45-5c5d89088ba1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:08.559535 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfc4208-96a5-4ea6-ad45-5c5d89088ba1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:08.561402 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfc4208-96a5-4ea6-ad45-5c5d89088ba1 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:08.562102 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555e76b0-03ea-4640-8ee6-8c3247480356 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5353f2af0f44d668e2a269647aa1dc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:08.562752 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555e76b0-03ea-4640-8ee6-8c3247480356 service glance] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:08.563027 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555e76b0-03ea-4640-8ee6-8c3247480356 service glance] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:08.563304 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555e76b0-03ea-4640-8ee6-8c3247480356 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:08.587651 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 838/1696] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:30:08 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:30:08.589549 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfc4208-96a5-4ea6-ad45-5c5d89088ba1 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:08.605231 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfc4208-96a5-4ea6-ad45-5c5d89088ba1 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:08.611932 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc516e9-c0b4-446e-ab2b-18958e4cb693 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:08.612946 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc516e9-c0b4-446e-ab2b-18958e4cb693 admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:08.613933 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc516e9-c0b4-446e-ab2b-18958e4cb693 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:08.614433 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc516e9-c0b4-446e-ab2b-18958e4cb693 admin admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:08.615922 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc516e9-c0b4-446e-ab2b-18958e4cb693 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:08.618120 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc516e9-c0b4-446e-ab2b-18958e4cb693 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:08.639311 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 839/1697] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:30:08 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:30:08.646478 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc516e9-c0b4-446e-ab2b-18958e4cb693 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:08.668127 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc516e9-c0b4-446e-ab2b-18958e4cb693 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:08.676654 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f41bacf-2dbe-463d-84e7-b970e1433f46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:08.681593 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f41bacf-2dbe-463d-84e7-b970e1433f46 admin admin] REQUEST_METHOD: `PUT` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:08.682114 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f41bacf-2dbe-463d-84e7-b970e1433f46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:08.682782 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f41bacf-2dbe-463d-84e7-b970e1433f46 admin admin] PATH_INFO: `/v3/projects/37217727cb1c49ffa919f8d8db787500/users/b82b53e3aa1d4104a44566408391bd29/roles/b494fedd00f24f0b84a529e8672a156a` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:08.684710 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555e76b0-03ea-4640-8ee6-8c3247480356 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:08.687730 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555e76b0-03ea-4640-8ee6-8c3247480356 service glance] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:08.712223 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 859/1698] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:30:08 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:30:08.724169 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f41bacf-2dbe-463d-84e7-b970e1433f46 admin admin] RBAC: Authorizing `identity:create_grant(project_id=37217727cb1c49ffa919f8d8db787500, user_id=b82b53e3aa1d4104a44566408391bd29, role_id=b494fedd00f24f0b84a529e8672a156a)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:08.732000 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f41bacf-2dbe-463d-84e7-b970e1433f46 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:08.754621 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 840/1699] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Nov 2 07:30:08 2022] PUT /identity/v3/projects/37217727cb1c49ffa919f8d8db787500/users/b82b53e3aa1d4104a44566408391bd29/roles/b494fedd00f24f0b84a529e8672a156a => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:30:08.768720 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81eeb116-ea4a-4b49-abbf-db5e6c3cbdd5 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:08.769757 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81eeb116-ea4a-4b49-abbf-db5e6c3cbdd5 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:08.770290 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81eeb116-ea4a-4b49-abbf-db5e6c3cbdd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:08.824633 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-81eeb116-ea4a-4b49-abbf-db5e6c3cbdd5 None None] MFA Rules not processed for user `b82b53e3aa1d4104a44566408391bd29`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:30:08.826549 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-81eeb116-ea4a-4b49-abbf-db5e6c3cbdd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:08.842057 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:30:08.930817 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 860/1700] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:30:08 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:30:09.315391 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f41bacf-2dbe-463d-84e7-b970e1433f46 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:09.317237 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-7f41bacf-2dbe-463d-84e7-b970e1433f46 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:09.387908 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81eeb116-ea4a-4b49-abbf-db5e6c3cbdd5 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:09.389465 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-81eeb116-ea4a-4b49-abbf-db5e6c3cbdd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:09.400641 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f41bacf-2dbe-463d-84e7-b970e1433f46 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:09.404321 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81eeb116-ea4a-4b49-abbf-db5e6c3cbdd5 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:09.405999 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdc38ad-26ba-4a6c-bd0d-d70c53f28928 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43bb326fb82e4d2c83fb2eaf91860dc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:09.406843 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdc38ad-26ba-4a6c-bd0d-d70c53f28928 service cinder] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:09.407271 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdc38ad-26ba-4a6c-bd0d-d70c53f28928 service cinder] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:09.407683 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdc38ad-26ba-4a6c-bd0d-d70c53f28928 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:09.408757 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27323431-c230-4e03-9d1d-e7667ef72ff1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43bb326fb82e4d2c83fb2eaf91860dc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:09.409351 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27323431-c230-4e03-9d1d-e7667ef72ff1 service cinder] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:09.409633 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27323431-c230-4e03-9d1d-e7667ef72ff1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:09.409910 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27323431-c230-4e03-9d1d-e7667ef72ff1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:09.412029 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-27323431-c230-4e03-9d1d-e7667ef72ff1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:09.418012 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdc38ad-26ba-4a6c-bd0d-d70c53f28928 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:09.420369 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdc38ad-26ba-4a6c-bd0d-d70c53f28928 service cinder] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:09.433109 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 841/1701] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:30:09 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:30:09.448902 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdc38ad-26ba-4a6c-bd0d-d70c53f28928 service cinder] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:09.505739 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27323431-c230-4e03-9d1d-e7667ef72ff1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:09.507137 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27323431-c230-4e03-9d1d-e7667ef72ff1 service cinder] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:09.521516 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 861/1702] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:30:09 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:30:09.533893 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdc38ad-26ba-4a6c-bd0d-d70c53f28928 service cinder] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:09.540898 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a73c7e7-36ae-4698-a1f1-0a2e55b55544 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:09.541551 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a73c7e7-36ae-4698-a1f1-0a2e55b55544 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:09.541836 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a73c7e7-36ae-4698-a1f1-0a2e55b55544 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:09.542308 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a73c7e7-36ae-4698-a1f1-0a2e55b55544 admin admin] PATH_INFO: `/v3/users/b82b53e3aa1d4104a44566408391bd29` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:09.543460 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a73c7e7-36ae-4698-a1f1-0a2e55b55544 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b82b53e3aa1d4104a44566408391bd29)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:09.546057 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a73c7e7-36ae-4698-a1f1-0a2e55b55544 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:09.603206 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=91,keystone:INSERT=4,keystone:DELETE=5 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:30:09.610733 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-1a73c7e7-36ae-4698-a1f1-0a2e55b55544 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b82b53e3aa1d4104a44566408391bd29'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:09.623143 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-1a73c7e7-36ae-4698-a1f1-0a2e55b55544 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b82b53e3aa1d4104a44566408391bd29'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:09.624726 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-1a73c7e7-36ae-4698-a1f1-0a2e55b55544 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b82b53e3aa1d4104a44566408391bd29'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:09.634715 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-1a73c7e7-36ae-4698-a1f1-0a2e55b55544 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b82b53e3aa1d4104a44566408391bd29'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:09.657203 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-1a73c7e7-36ae-4698-a1f1-0a2e55b55544 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b82b53e3aa1d4104a44566408391bd29'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:09.665764 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 842/1703] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Nov 2 07:30:09 2022] DELETE /identity/v3/users/b82b53e3aa1d4104a44566408391bd29 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 07:30:09.670480 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c278f779-c3b1-4c3b-86b7-39a627f4c764 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:09.670896 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c278f779-c3b1-4c3b-86b7-39a627f4c764 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:09.671273 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c278f779-c3b1-4c3b-86b7-39a627f4c764 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:09.691601 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-c278f779-c3b1-4c3b-86b7-39a627f4c764 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:30:09.692931 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-c278f779-c3b1-4c3b-86b7-39a627f4c764 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:09.755880 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 862/1704] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:30:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:30:09.766605 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a73c7e7-36ae-4698-a1f1-0a2e55b55544 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:09.768171 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-1a73c7e7-36ae-4698-a1f1-0a2e55b55544 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:09.827996 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a73c7e7-36ae-4698-a1f1-0a2e55b55544 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:09.831615 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e39c49-7646-449a-8f8b-3112dd6a371a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3ec6ecb85f94ef59762877d9ea33ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:09.832216 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e39c49-7646-449a-8f8b-3112dd6a371a service neutron] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:09.832491 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e39c49-7646-449a-8f8b-3112dd6a371a service neutron] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:09.832757 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e39c49-7646-449a-8f8b-3112dd6a371a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:09.842617 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e39c49-7646-449a-8f8b-3112dd6a371a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:09.844033 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e39c49-7646-449a-8f8b-3112dd6a371a service neutron] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:09.853645 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 843/1705] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:30:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:30:10.157572 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c278f779-c3b1-4c3b-86b7-39a627f4c764 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:10.162605 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-c278f779-c3b1-4c3b-86b7-39a627f4c764 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:10.175523 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c278f779-c3b1-4c3b-86b7-39a627f4c764 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:10.179791 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63ab634-8145-4df1-b6a9-4d9c9a3bf25c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:10.180424 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63ab634-8145-4df1-b6a9-4d9c9a3bf25c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:10.180697 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63ab634-8145-4df1-b6a9-4d9c9a3bf25c admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:10.180964 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63ab634-8145-4df1-b6a9-4d9c9a3bf25c admin admin] PATH_INFO: `/v3/projects/37217727cb1c49ffa919f8d8db787500` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:10.182066 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63ab634-8145-4df1-b6a9-4d9c9a3bf25c admin admin] RBAC: Authorizing `identity:delete_project(project_id=37217727cb1c49ffa919f8d8db787500)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:10.184032 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63ab634-8145-4df1-b6a9-4d9c9a3bf25c admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:10.235758 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:30:10.238772 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-a63ab634-8145-4df1-b6a9-4d9c9a3bf25c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '37217727cb1c49ffa919f8d8db787500', 'request_id': 'req-a63ab634-8145-4df1-b6a9-4d9c9a3bf25c'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:10.247177 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-a63ab634-8145-4df1-b6a9-4d9c9a3bf25c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '37217727cb1c49ffa919f8d8db787500', 'request_id': 'req-a63ab634-8145-4df1-b6a9-4d9c9a3bf25c'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:10.252863 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-a63ab634-8145-4df1-b6a9-4d9c9a3bf25c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '37217727cb1c49ffa919f8d8db787500', 'request_id': 'req-a63ab634-8145-4df1-b6a9-4d9c9a3bf25c'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:10.265195 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-a63ab634-8145-4df1-b6a9-4d9c9a3bf25c admin admin] The token cache is being invalidate because project 37217727cb1c49ffa919f8d8db787500 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:30:10.265407 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-a63ab634-8145-4df1-b6a9-4d9c9a3bf25c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:10.266024 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-a63ab634-8145-4df1-b6a9-4d9c9a3bf25c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:10.267937 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 863/1706] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Nov 2 07:30:10 2022] DELETE /identity/v3/projects/37217727cb1c49ffa919f8d8db787500 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:30:10.273443 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e5053b-8c15-4881-9dc3-33c3814ceb8c None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:10.273725 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e5053b-8c15-4881-9dc3-33c3814ceb8c None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:10.273992 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e5053b-8c15-4881-9dc3-33c3814ceb8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:10.293617 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-c4e5053b-8c15-4881-9dc3-33c3814ceb8c None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:30:10.294840 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-c4e5053b-8c15-4881-9dc3-33c3814ceb8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:10.353981 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 844/1707] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:30:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:30:10.362496 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ace0f5-0a52-4cab-b593-4ae521262d96 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:10.362806 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ace0f5-0a52-4cab-b593-4ae521262d96 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:10.363082 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ace0f5-0a52-4cab-b593-4ae521262d96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:10.383631 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-e3ace0f5-0a52-4cab-b593-4ae521262d96 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:30:10.384916 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-e3ace0f5-0a52-4cab-b593-4ae521262d96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:10.396391 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 864/1708] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:30:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:30:10.398353 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e5053b-8c15-4881-9dc3-33c3814ceb8c None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:10.408707 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e5053b-8c15-4881-9dc3-33c3814ceb8c None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:10.413436 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe27c987-eabd-4c48-8ea5-61bf1e32bcd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:10.414114 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe27c987-eabd-4c48-8ea5-61bf1e32bcd7 admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:10.414499 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe27c987-eabd-4c48-8ea5-61bf1e32bcd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:10.414841 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe27c987-eabd-4c48-8ea5-61bf1e32bcd7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:10.415620 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe27c987-eabd-4c48-8ea5-61bf1e32bcd7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:30:10.415910 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe27c987-eabd-4c48-8ea5-61bf1e32bcd7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:10.418117 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe27c987-eabd-4c48-8ea5-61bf1e32bcd7 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:10.433699 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 845/1709] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Nov 2 07:30:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:30:10.438179 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ace0f5-0a52-4cab-b593-4ae521262d96 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:10.447931 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ace0f5-0a52-4cab-b593-4ae521262d96 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:10.452611 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5624c73-5867-481d-bd22-f059b5d391d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:10.453212 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5624c73-5867-481d-bd22-f059b5d391d4 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:10.453489 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5624c73-5867-481d-bd22-f059b5d391d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:10.453758 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5624c73-5867-481d-bd22-f059b5d391d4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:10.454581 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5624c73-5867-481d-bd22-f059b5d391d4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:10.456739 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5624c73-5867-481d-bd22-f059b5d391d4 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:10.483990 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 865/1710] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Nov 2 07:30:10 2022] POST /identity/v3/projects => generated 385 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:30:10.488167 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe27c987-eabd-4c48-8ea5-61bf1e32bcd7 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:10.499023 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe27c987-eabd-4c48-8ea5-61bf1e32bcd7 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:10.546341 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:30:10.562685 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911ab790-6ac8-43c4-9c38-fa541c78acb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:10.563551 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911ab790-6ac8-43c4-9c38-fa541c78acb4 admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:10.563859 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911ab790-6ac8-43c4-9c38-fa541c78acb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:10.564337 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911ab790-6ac8-43c4-9c38-fa541c78acb4 admin admin] PATH_INFO: `/v3/users` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:10.565437 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911ab790-6ac8-43c4-9c38-fa541c78acb4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:10.567831 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911ab790-6ac8-43c4-9c38-fa541c78acb4 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:10.593970 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 846/1711] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Nov 2 07:30:10 2022] POST /identity/v3/users => generated 293 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:30:10.597928 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5624c73-5867-481d-bd22-f059b5d391d4 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:10.611393 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5624c73-5867-481d-bd22-f059b5d391d4 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:10.619324 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5a9985-6512-4531-8cc3-ed4cd88e49fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:10.620211 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5a9985-6512-4531-8cc3-ed4cd88e49fb admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:10.620564 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5a9985-6512-4531-8cc3-ed4cd88e49fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:10.621198 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5a9985-6512-4531-8cc3-ed4cd88e49fb admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:10.622091 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5a9985-6512-4531-8cc3-ed4cd88e49fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:10.623921 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5a9985-6512-4531-8cc3-ed4cd88e49fb admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:10.650063 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 866/1712] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:30:10 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:30:10.654669 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911ab790-6ac8-43c4-9c38-fa541c78acb4 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:10.665457 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911ab790-6ac8-43c4-9c38-fa541c78acb4 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:10.673029 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a293783c-65f0-4bd5-b35f-637c7f222fcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:10.673029 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a293783c-65f0-4bd5-b35f-637c7f222fcc admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:10.673251 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a293783c-65f0-4bd5-b35f-637c7f222fcc admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:10.673400 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a293783c-65f0-4bd5-b35f-637c7f222fcc admin admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:10.678585 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a293783c-65f0-4bd5-b35f-637c7f222fcc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:10.678585 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a293783c-65f0-4bd5-b35f-637c7f222fcc admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:10.690308 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 847/1713] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:30:10 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:30:10.694437 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5a9985-6512-4531-8cc3-ed4cd88e49fb admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:10.705286 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5a9985-6512-4531-8cc3-ed4cd88e49fb admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:10.710195 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dfe242-6b67-4b43-82da-aeaeb6acd763 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:10.711050 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dfe242-6b67-4b43-82da-aeaeb6acd763 admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:10.711393 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dfe242-6b67-4b43-82da-aeaeb6acd763 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:10.711730 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dfe242-6b67-4b43-82da-aeaeb6acd763 admin admin] PATH_INFO: `/v3/projects/ac8ef1a11bce4dd3b99fa75f82218c37/users/6eba86a1c4634b32ad0c7c65bb1f239c/roles/b494fedd00f24f0b84a529e8672a156a` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:10.740383 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58dfe242-6b67-4b43-82da-aeaeb6acd763 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac8ef1a11bce4dd3b99fa75f82218c37, user_id=6eba86a1c4634b32ad0c7c65bb1f239c, role_id=b494fedd00f24f0b84a529e8672a156a)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:10.743368 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58dfe242-6b67-4b43-82da-aeaeb6acd763 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:10.762429 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 867/1714] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Nov 2 07:30:10 2022] PUT /identity/v3/projects/ac8ef1a11bce4dd3b99fa75f82218c37/users/6eba86a1c4634b32ad0c7c65bb1f239c/roles/b494fedd00f24f0b84a529e8672a156a => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:30:10.779537 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e676c3b0-bb5c-43e8-a4a9-ea6b6f30d856 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:10.779895 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e676c3b0-bb5c-43e8-a4a9-ea6b6f30d856 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:10.780175 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e676c3b0-bb5c-43e8-a4a9-ea6b6f30d856 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:10.808575 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-e676c3b0-bb5c-43e8-a4a9-ea6b6f30d856 None None] MFA Rules not processed for user `6eba86a1c4634b32ad0c7c65bb1f239c`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:30:10.809708 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-e676c3b0-bb5c-43e8-a4a9-ea6b6f30d856 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:10.876648 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 848/1715] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:30:10 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:30:10.888376 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dfe242-6b67-4b43-82da-aeaeb6acd763 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:10.890135 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-58dfe242-6b67-4b43-82da-aeaeb6acd763 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:10.946500 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dfe242-6b67-4b43-82da-aeaeb6acd763 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:10.950702 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a93de2-90db-4547-acd4-6fd9dbe387df service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43bb326fb82e4d2c83fb2eaf91860dc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:10.951321 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a93de2-90db-4547-acd4-6fd9dbe387df service cinder] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:10.951592 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a93de2-90db-4547-acd4-6fd9dbe387df service cinder] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:10.951859 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a93de2-90db-4547-acd4-6fd9dbe387df service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:10.960503 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a93de2-90db-4547-acd4-6fd9dbe387df service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:10.961915 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a93de2-90db-4547-acd4-6fd9dbe387df service cinder] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:10.971851 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 868/1716] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:30:10 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:30:17.863750 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e676c3b0-bb5c-43e8-a4a9-ea6b6f30d856 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:17.934025 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e676c3b0-bb5c-43e8-a4a9-ea6b6f30d856 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:17.938643 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebf4c92-0f31-4061-beb7-494d8c805a2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:17.939273 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebf4c92-0f31-4061-beb7-494d8c805a2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:17.939578 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebf4c92-0f31-4061-beb7-494d8c805a2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:17.939848 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebf4c92-0f31-4061-beb7-494d8c805a2c admin admin] PATH_INFO: `/v3/users/6eba86a1c4634b32ad0c7c65bb1f239c` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:17.940944 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebf4c92-0f31-4061-beb7-494d8c805a2c admin admin] RBAC: Authorizing `identity:delete_user(user_id=6eba86a1c4634b32ad0c7c65bb1f239c)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:17.942946 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebf4c92-0f31-4061-beb7-494d8c805a2c admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:17.993121 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-bebf4c92-0f31-4061-beb7-494d8c805a2c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6eba86a1c4634b32ad0c7c65bb1f239c'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:18.001793 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-bebf4c92-0f31-4061-beb7-494d8c805a2c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6eba86a1c4634b32ad0c7c65bb1f239c'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:18.002575 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-bebf4c92-0f31-4061-beb7-494d8c805a2c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6eba86a1c4634b32ad0c7c65bb1f239c'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:18.004720 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:30:18.011225 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-bebf4c92-0f31-4061-beb7-494d8c805a2c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6eba86a1c4634b32ad0c7c65bb1f239c'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:18.025408 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-bebf4c92-0f31-4061-beb7-494d8c805a2c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6eba86a1c4634b32ad0c7c65bb1f239c'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:18.031794 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 849/1717] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Nov 2 07:30:17 2022] DELETE /identity/v3/users/6eba86a1c4634b32ad0c7c65bb1f239c => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:30:18.036289 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42fa873-548d-4347-b438-8963714d2b8c None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:18.036762 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42fa873-548d-4347-b438-8963714d2b8c None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:18.036867 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42fa873-548d-4347-b438-8963714d2b8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:18.055419 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-b42fa873-548d-4347-b438-8963714d2b8c None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:30:18.056616 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-b42fa873-548d-4347-b438-8963714d2b8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:18.120739 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 869/1718] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:30:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:30:18.131572 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebf4c92-0f31-4061-beb7-494d8c805a2c admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:18.133136 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-bebf4c92-0f31-4061-beb7-494d8c805a2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:18.198710 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebf4c92-0f31-4061-beb7-494d8c805a2c admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:18.202859 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b484180-3cd4-4ce3-bd9f-cbdeccdee882 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3ec6ecb85f94ef59762877d9ea33ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:18.203648 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b484180-3cd4-4ce3-bd9f-cbdeccdee882 service neutron] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:18.203914 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b484180-3cd4-4ce3-bd9f-cbdeccdee882 service neutron] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:18.204187 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b484180-3cd4-4ce3-bd9f-cbdeccdee882 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:18.213797 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b484180-3cd4-4ce3-bd9f-cbdeccdee882 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:18.215427 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b484180-3cd4-4ce3-bd9f-cbdeccdee882 service neutron] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:18.225942 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 850/1719] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:30:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:30:18.495726 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42fa873-548d-4347-b438-8963714d2b8c None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:18.497603 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-b42fa873-548d-4347-b438-8963714d2b8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:18.519025 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42fa873-548d-4347-b438-8963714d2b8c None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:18.525543 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda955a6-6e17-4615-85e4-fd1141041a3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:18.526543 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda955a6-6e17-4615-85e4-fd1141041a3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:18.526947 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda955a6-6e17-4615-85e4-fd1141041a3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:18.527821 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda955a6-6e17-4615-85e4-fd1141041a3a admin admin] PATH_INFO: `/v3/users/5605da3621a446c598227d84d80cc45c` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:18.555854 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda955a6-6e17-4615-85e4-fd1141041a3a admin admin] RBAC: Authorizing `identity:delete_user(user_id=5605da3621a446c598227d84d80cc45c)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:18.555854 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda955a6-6e17-4615-85e4-fd1141041a3a admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:18.562038 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:DELETE=4,keystone:SELECT=93,keystone:INSERT=3 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:30:18.663848 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-eda955a6-6e17-4615-85e4-fd1141041a3a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5605da3621a446c598227d84d80cc45c'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:18.673012 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-eda955a6-6e17-4615-85e4-fd1141041a3a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5605da3621a446c598227d84d80cc45c'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:18.673700 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-eda955a6-6e17-4615-85e4-fd1141041a3a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5605da3621a446c598227d84d80cc45c'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:18.684805 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-eda955a6-6e17-4615-85e4-fd1141041a3a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5605da3621a446c598227d84d80cc45c'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:18.684805 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-eda955a6-6e17-4615-85e4-fd1141041a3a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5605da3621a446c598227d84d80cc45c'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:18.687320 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b484180-3cd4-4ce3-bd9f-cbdeccdee882 service neutron] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:18.689367 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-6b484180-3cd4-4ce3-bd9f-cbdeccdee882 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:18.710763 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b484180-3cd4-4ce3-bd9f-cbdeccdee882 service neutron] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:18.716011 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d7294b-4237-4ecb-b347-353be67c3296 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:18.716673 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d7294b-4237-4ecb-b347-353be67c3296 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:18.716970 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d7294b-4237-4ecb-b347-353be67c3296 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:18.717260 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d7294b-4237-4ecb-b347-353be67c3296 admin admin] PATH_INFO: `/v3/projects/ac8ef1a11bce4dd3b99fa75f82218c37` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:18.718420 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d7294b-4237-4ecb-b347-353be67c3296 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ac8ef1a11bce4dd3b99fa75f82218c37)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:18.720486 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d7294b-4237-4ecb-b347-353be67c3296 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:18.723450 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 870/1720] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Nov 2 07:30:18 2022] DELETE /identity/v3/users/5605da3621a446c598227d84d80cc45c => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:30:18.731563 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e4110f-a8f5-480e-84ab-8ea8f86b05d9 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:18.731853 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e4110f-a8f5-480e-84ab-8ea8f86b05d9 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:18.732668 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e4110f-a8f5-480e-84ab-8ea8f86b05d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:18.763532 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-e4e4110f-a8f5-480e-84ab-8ea8f86b05d9 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:30:18.765049 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-e4e4110f-a8f5-480e-84ab-8ea8f86b05d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:18.792361 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-c9d7294b-4237-4ecb-b347-353be67c3296 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ac8ef1a11bce4dd3b99fa75f82218c37', 'request_id': 'req-c9d7294b-4237-4ecb-b347-353be67c3296'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:18.800858 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-c9d7294b-4237-4ecb-b347-353be67c3296 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ac8ef1a11bce4dd3b99fa75f82218c37', 'request_id': 'req-c9d7294b-4237-4ecb-b347-353be67c3296'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:18.816529 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-c9d7294b-4237-4ecb-b347-353be67c3296 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ac8ef1a11bce4dd3b99fa75f82218c37', 'request_id': 'req-c9d7294b-4237-4ecb-b347-353be67c3296'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:18.837766 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-c9d7294b-4237-4ecb-b347-353be67c3296 admin admin] The token cache is being invalidate because project ac8ef1a11bce4dd3b99fa75f82218c37 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81501) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:30:18.838497 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-c9d7294b-4237-4ecb-b347-353be67c3296 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:18.839663 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-c9d7294b-4237-4ecb-b347-353be67c3296 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:18.842126 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 851/1721] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Nov 2 07:30:18 2022] DELETE /identity/v3/projects/ac8ef1a11bce4dd3b99fa75f82218c37 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 07:30:18.850789 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c624ce98-0e1d-4c96-96c3-cad8f075d3e5 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:18.854572 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 871/1722] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:30:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:30:18.854978 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c624ce98-0e1d-4c96-96c3-cad8f075d3e5 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:18.855702 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c624ce98-0e1d-4c96-96c3-cad8f075d3e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:18.866116 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e4110f-a8f5-480e-84ab-8ea8f86b05d9 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:18.870919 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-e4e4110f-a8f5-480e-84ab-8ea8f86b05d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:18.892129 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-c624ce98-0e1d-4c96-96c3-cad8f075d3e5 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:30:18.894109 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-c624ce98-0e1d-4c96-96c3-cad8f075d3e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:18.973744 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:30:18.986121 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e4110f-a8f5-480e-84ab-8ea8f86b05d9 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:18.988447 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 852/1723] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:30:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:30:18.998789 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848046b8-fcb1-40bd-b755-7bc1fc5af745 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:18.999181 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848046b8-fcb1-40bd-b755-7bc1fc5af745 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:18.999465 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848046b8-fcb1-40bd-b755-7bc1fc5af745 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:19.026457 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-848046b8-fcb1-40bd-b755-7bc1fc5af745 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:30:19.027676 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-848046b8-fcb1-40bd-b755-7bc1fc5af745 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:19.039606 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 853/1724] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:30:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:30:19.043085 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848046b8-fcb1-40bd-b755-7bc1fc5af745 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:19.057027 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848046b8-fcb1-40bd-b755-7bc1fc5af745 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:19.063901 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30a4bc5-5dbf-4ed6-bb0a-48160b2ec6d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:19.064594 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30a4bc5-5dbf-4ed6-bb0a-48160b2ec6d1 admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:19.064881 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30a4bc5-5dbf-4ed6-bb0a-48160b2ec6d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:19.066439 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30a4bc5-5dbf-4ed6-bb0a-48160b2ec6d1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:19.067446 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30a4bc5-5dbf-4ed6-bb0a-48160b2ec6d1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:30:19.067894 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30a4bc5-5dbf-4ed6-bb0a-48160b2ec6d1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:19.069512 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:30:19.069980 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30a4bc5-5dbf-4ed6-bb0a-48160b2ec6d1 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:19.087242 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 854/1725] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Nov 2 07:30:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:30:19.088697 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f4a9c3-c4b4-4704-bff8-7de26c6c3a27 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3ec6ecb85f94ef59762877d9ea33ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:19.088978 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30a4bc5-5dbf-4ed6-bb0a-48160b2ec6d1 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:19.089407 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f4a9c3-c4b4-4704-bff8-7de26c6c3a27 service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:19.089689 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f4a9c3-c4b4-4704-bff8-7de26c6c3a27 service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:19.089991 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f4a9c3-c4b4-4704-bff8-7de26c6c3a27 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:19.101235 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30a4bc5-5dbf-4ed6-bb0a-48160b2ec6d1 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:19.101838 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f4a9c3-c4b4-4704-bff8-7de26c6c3a27 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:19.103437 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f4a9c3-c4b4-4704-bff8-7de26c6c3a27 service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:19.106382 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea03a2d6-267d-4dbb-8510-5ee8f3790336 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:19.107067 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea03a2d6-267d-4dbb-8510-5ee8f3790336 admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:19.107939 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea03a2d6-267d-4dbb-8510-5ee8f3790336 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:19.108442 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea03a2d6-267d-4dbb-8510-5ee8f3790336 admin admin] PATH_INFO: `/v3/projects` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:19.109070 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea03a2d6-267d-4dbb-8510-5ee8f3790336 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:19.111124 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea03a2d6-267d-4dbb-8510-5ee8f3790336 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:19.115011 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 872/1726] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:30:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 249 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 07:30:19.147045 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 855/1727] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Nov 2 07:30:19 2022] POST /identity/v3/projects => generated 377 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:30:19.151180 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f4a9c3-c4b4-4704-bff8-7de26c6c3a27 service neutron] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:19.161260 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f4a9c3-c4b4-4704-bff8-7de26c6c3a27 service neutron] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:19.259421 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86834f7-098f-4205-8059-334f23b03736 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:19.260954 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86834f7-098f-4205-8059-334f23b03736 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:19.260954 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86834f7-098f-4205-8059-334f23b03736 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:19.261206 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86834f7-098f-4205-8059-334f23b03736 admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:19.261897 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86834f7-098f-4205-8059-334f23b03736 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:19.265631 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86834f7-098f-4205-8059-334f23b03736 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:19.296612 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 873/1728] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Nov 2 07:30:19 2022] POST /identity/v3/users => generated 289 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 07:30:19.300934 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea03a2d6-267d-4dbb-8510-5ee8f3790336 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:19.334336 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea03a2d6-267d-4dbb-8510-5ee8f3790336 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:19.344032 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0938135-45a1-4f4b-84d9-dd42451d2f85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:19.344747 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0938135-45a1-4f4b-84d9-dd42451d2f85 admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:19.345106 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0938135-45a1-4f4b-84d9-dd42451d2f85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:19.345314 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0938135-45a1-4f4b-84d9-dd42451d2f85 admin admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:19.346021 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0938135-45a1-4f4b-84d9-dd42451d2f85 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:19.347738 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0938135-45a1-4f4b-84d9-dd42451d2f85 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:19.362915 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 856/1729] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:30:19 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:30:19.366586 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86834f7-098f-4205-8059-334f23b03736 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:19.381540 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86834f7-098f-4205-8059-334f23b03736 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:19.389230 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348ee407-bafb-485c-b734-e2a97995013b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:19.389976 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348ee407-bafb-485c-b734-e2a97995013b admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:19.390323 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348ee407-bafb-485c-b734-e2a97995013b admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:19.390987 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348ee407-bafb-485c-b734-e2a97995013b admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:19.391639 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348ee407-bafb-485c-b734-e2a97995013b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:19.393271 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348ee407-bafb-485c-b734-e2a97995013b admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:19.407152 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 874/1730] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:30:19 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:30:19.410994 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0938135-45a1-4f4b-84d9-dd42451d2f85 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:19.422083 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0938135-45a1-4f4b-84d9-dd42451d2f85 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:19.436892 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d826d4-ceae-47dc-b4c0-286b20141ba8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:19.437939 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d826d4-ceae-47dc-b4c0-286b20141ba8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:19.438220 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d826d4-ceae-47dc-b4c0-286b20141ba8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:19.438976 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d826d4-ceae-47dc-b4c0-286b20141ba8 admin admin] PATH_INFO: `/v3/projects/566514b413e24aaebe69c11ce2159ac9/users/fe26aea6bb9d4dcebf046da719b2c244/roles/b494fedd00f24f0b84a529e8672a156a` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:19.459993 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348ee407-bafb-485c-b734-e2a97995013b admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:19.462155 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-348ee407-bafb-485c-b734-e2a97995013b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:19.466194 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d826d4-ceae-47dc-b4c0-286b20141ba8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=566514b413e24aaebe69c11ce2159ac9, user_id=fe26aea6bb9d4dcebf046da719b2c244, role_id=b494fedd00f24f0b84a529e8672a156a)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:19.469136 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d826d4-ceae-47dc-b4c0-286b20141ba8 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:19.480957 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348ee407-bafb-485c-b734-e2a97995013b admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:19.489035 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a770a53-db12-4b39-be78-f60719acf55d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:19.489558 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a770a53-db12-4b39-be78-f60719acf55d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:19.490646 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a770a53-db12-4b39-be78-f60719acf55d admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:19.490941 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a770a53-db12-4b39-be78-f60719acf55d admin admin] PATH_INFO: `/v3/projects/91af8acabef14e55b88cddae6628cada` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:19.493044 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a770a53-db12-4b39-be78-f60719acf55d admin admin] RBAC: Authorizing `identity:delete_project(project_id=91af8acabef14e55b88cddae6628cada)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:19.495875 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a770a53-db12-4b39-be78-f60719acf55d admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:19.500270 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 857/1731] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Nov 2 07:30:19 2022] PUT /identity/v3/projects/566514b413e24aaebe69c11ce2159ac9/users/fe26aea6bb9d4dcebf046da719b2c244/roles/b494fedd00f24f0b84a529e8672a156a => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 07:30:19.505815 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af21f10b-2284-4ac8-a10f-d0d222fba79c None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:19.507230 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af21f10b-2284-4ac8-a10f-d0d222fba79c None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:19.507725 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af21f10b-2284-4ac8-a10f-d0d222fba79c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:19.532189 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-af21f10b-2284-4ac8-a10f-d0d222fba79c None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:30:19.533348 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-af21f10b-2284-4ac8-a10f-d0d222fba79c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:19.564778 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-1a770a53-db12-4b39-be78-f60719acf55d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '91af8acabef14e55b88cddae6628cada', 'request_id': 'req-1a770a53-db12-4b39-be78-f60719acf55d'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:19.574476 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-1a770a53-db12-4b39-be78-f60719acf55d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '91af8acabef14e55b88cddae6628cada', 'request_id': 'req-1a770a53-db12-4b39-be78-f60719acf55d'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:19.583070 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-1a770a53-db12-4b39-be78-f60719acf55d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '91af8acabef14e55b88cddae6628cada', 'request_id': 'req-1a770a53-db12-4b39-be78-f60719acf55d'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:19.599615 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-1a770a53-db12-4b39-be78-f60719acf55d admin admin] The token cache is being invalidate because project 91af8acabef14e55b88cddae6628cada was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:30:19.599952 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-1a770a53-db12-4b39-be78-f60719acf55d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:19.600734 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-1a770a53-db12-4b39-be78-f60719acf55d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:19.602810 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 875/1732] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Nov 2 07:30:19 2022] DELETE /identity/v3/projects/91af8acabef14e55b88cddae6628cada => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 07:30:19.607778 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a770a53-db12-4b39-be78-f60719acf55d admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:19.611491 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 858/1733] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:30:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:30:19.619611 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-1a770a53-db12-4b39-be78-f60719acf55d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:19.621943 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af21f10b-2284-4ac8-a10f-d0d222fba79c None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:19.623889 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-af21f10b-2284-4ac8-a10f-d0d222fba79c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:19.695683 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:30:19.715040 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af21f10b-2284-4ac8-a10f-d0d222fba79c None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:19.722575 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a770a53-db12-4b39-be78-f60719acf55d admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:19.731216 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd10cc4-ebf8-435e-8363-37b651ddd204 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:19.732387 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd10cc4-ebf8-435e-8363-37b651ddd204 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:19.732807 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd10cc4-ebf8-435e-8363-37b651ddd204 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:19.733255 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd10cc4-ebf8-435e-8363-37b651ddd204 admin admin] PATH_INFO: `/v3/users/57b832efd7d94a64beb1e37514a07367` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:19.734732 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd10cc4-ebf8-435e-8363-37b651ddd204 admin admin] RBAC: Authorizing `identity:delete_user(user_id=57b832efd7d94a64beb1e37514a07367)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:19.736910 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd10cc4-ebf8-435e-8363-37b651ddd204 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:19.773655 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:30:19.788792 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffea1a69-1ac5-4638-bd61-2d8d943c2f10 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3ec6ecb85f94ef59762877d9ea33ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:19.789572 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffea1a69-1ac5-4638-bd61-2d8d943c2f10 service neutron] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:19.789946 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffea1a69-1ac5-4638-bd61-2d8d943c2f10 service neutron] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:19.790313 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffea1a69-1ac5-4638-bd61-2d8d943c2f10 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:19.798191 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-1dd10cc4-ebf8-435e-8363-37b651ddd204 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '57b832efd7d94a64beb1e37514a07367'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:19.803023 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffea1a69-1ac5-4638-bd61-2d8d943c2f10 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:19.804444 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffea1a69-1ac5-4638-bd61-2d8d943c2f10 service neutron] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:19.807852 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-1dd10cc4-ebf8-435e-8363-37b651ddd204 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '57b832efd7d94a64beb1e37514a07367'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:19.808812 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-1dd10cc4-ebf8-435e-8363-37b651ddd204 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '57b832efd7d94a64beb1e37514a07367'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:19.816038 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 859/1734] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:30:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 07:30:19.831021 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-1dd10cc4-ebf8-435e-8363-37b651ddd204 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '57b832efd7d94a64beb1e37514a07367'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:19.831866 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-1dd10cc4-ebf8-435e-8363-37b651ddd204 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '57b832efd7d94a64beb1e37514a07367'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:19.849988 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 876/1735] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Nov 2 07:30:19 2022] DELETE /identity/v3/users/57b832efd7d94a64beb1e37514a07367 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 07:30:20.165149 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffea1a69-1ac5-4638-bd61-2d8d943c2f10 service neutron] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:20.167259 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-ffea1a69-1ac5-4638-bd61-2d8d943c2f10 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:20.185183 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffea1a69-1ac5-4638-bd61-2d8d943c2f10 service neutron] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:20.200753 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd10cc4-ebf8-435e-8363-37b651ddd204 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:20.205713 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-1dd10cc4-ebf8-435e-8363-37b651ddd204 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:20.319684 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd10cc4-ebf8-435e-8363-37b651ddd204 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:20.326304 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1c9d0a-019a-43c6-ab22-a73a38125376 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:20.328205 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1c9d0a-019a-43c6-ab22-a73a38125376 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:20.328706 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1c9d0a-019a-43c6-ab22-a73a38125376 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:20.329191 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1c9d0a-019a-43c6-ab22-a73a38125376 admin admin] PATH_INFO: `/v3/projects/595ae4b6a60c42d1939420954c75d1eb` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:20.330995 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1c9d0a-019a-43c6-ab22-a73a38125376 admin admin] RBAC: Authorizing `identity:delete_project(project_id=595ae4b6a60c42d1939420954c75d1eb)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:20.332317 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24613acd-b820-40e5-9540-1f347de6b3dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:20.334489 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24613acd-b820-40e5-9540-1f347de6b3dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:20.334489 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24613acd-b820-40e5-9540-1f347de6b3dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:20.334489 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24613acd-b820-40e5-9540-1f347de6b3dc admin admin] PATH_INFO: `/v3/users/b128774a38964c0d9104318572433143` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:20.334489 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24613acd-b820-40e5-9540-1f347de6b3dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=b128774a38964c0d9104318572433143)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:20.336656 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24613acd-b820-40e5-9540-1f347de6b3dc admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:20.337461 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1c9d0a-019a-43c6-ab22-a73a38125376 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:20.408531 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-24613acd-b820-40e5-9540-1f347de6b3dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b128774a38964c0d9104318572433143'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:20.416800 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-0b1c9d0a-019a-43c6-ab22-a73a38125376 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '595ae4b6a60c42d1939420954c75d1eb', 'request_id': 'req-0b1c9d0a-019a-43c6-ab22-a73a38125376'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:20.421984 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-24613acd-b820-40e5-9540-1f347de6b3dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b128774a38964c0d9104318572433143'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:20.423928 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-24613acd-b820-40e5-9540-1f347de6b3dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b128774a38964c0d9104318572433143'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:20.429457 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-0b1c9d0a-019a-43c6-ab22-a73a38125376 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '595ae4b6a60c42d1939420954c75d1eb', 'request_id': 'req-0b1c9d0a-019a-43c6-ab22-a73a38125376'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:20.438679 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-24613acd-b820-40e5-9540-1f347de6b3dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b128774a38964c0d9104318572433143'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:20.441154 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-0b1c9d0a-019a-43c6-ab22-a73a38125376 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '595ae4b6a60c42d1939420954c75d1eb', 'request_id': 'req-0b1c9d0a-019a-43c6-ab22-a73a38125376'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:20.447546 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:30:20.456469 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-24613acd-b820-40e5-9540-1f347de6b3dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b128774a38964c0d9104318572433143'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:20.467624 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 860/1736] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Nov 2 07:30:20 2022] DELETE /identity/v3/users/b128774a38964c0d9104318572433143 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:30:20.469173 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0ac404-2d31-49dd-9c5c-e16ccddaaf55 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:20.469524 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0ac404-2d31-49dd-9c5c-e16ccddaaf55 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:20.469791 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0ac404-2d31-49dd-9c5c-e16ccddaaf55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:20.476081 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-0b1c9d0a-019a-43c6-ab22-a73a38125376 admin admin] The token cache is being invalidate because project 595ae4b6a60c42d1939420954c75d1eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:30:20.476647 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-0b1c9d0a-019a-43c6-ab22-a73a38125376 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:20.477799 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-0b1c9d0a-019a-43c6-ab22-a73a38125376 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:20.479952 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 877/1737] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Nov 2 07:30:20 2022] DELETE /identity/v3/projects/595ae4b6a60c42d1939420954c75d1eb => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 07:30:20.485237 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee74ce4c-554c-45a8-875c-90f97254275f None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:20.485645 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee74ce4c-554c-45a8-875c-90f97254275f None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:20.486027 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee74ce4c-554c-45a8-875c-90f97254275f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:20.491378 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-4d0ac404-2d31-49dd-9c5c-e16ccddaaf55 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:30:20.493177 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-4d0ac404-2d31-49dd-9c5c-e16ccddaaf55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:20.520855 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-ee74ce4c-554c-45a8-875c-90f97254275f None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:30:20.524088 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-ee74ce4c-554c-45a8-875c-90f97254275f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:20.594013 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 861/1738] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:30:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:30:20.604016 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0ac404-2d31-49dd-9c5c-e16ccddaaf55 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:20.606048 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-4d0ac404-2d31-49dd-9c5c-e16ccddaaf55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:20.616135 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:30:20.672251 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 878/1739] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:30:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:30:20.679109 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b14339-9324-4850-a23d-e64a0a4fdb12 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:20.679599 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b14339-9324-4850-a23d-e64a0a4fdb12 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:20.679982 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b14339-9324-4850-a23d-e64a0a4fdb12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:20.729328 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-34b14339-9324-4850-a23d-e64a0a4fdb12 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:30:20.731679 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-34b14339-9324-4850-a23d-e64a0a4fdb12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:20.751336 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0ac404-2d31-49dd-9c5c-e16ccddaaf55 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:20.756247 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 879/1740] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:30:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:30:20.757924 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b14339-9324-4850-a23d-e64a0a4fdb12 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:20.760182 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26cfe3e-f517-4140-b66e-9f3795349ac7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3ec6ecb85f94ef59762877d9ea33ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:20.761608 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26cfe3e-f517-4140-b66e-9f3795349ac7 service neutron] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:20.763167 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26cfe3e-f517-4140-b66e-9f3795349ac7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:20.764374 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26cfe3e-f517-4140-b66e-9f3795349ac7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:20.771387 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b14339-9324-4850-a23d-e64a0a4fdb12 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:20.779142 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e254431-4f54-4460-8ae6-9d9a5ab30af1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:20.779744 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e254431-4f54-4460-8ae6-9d9a5ab30af1 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:20.781644 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e254431-4f54-4460-8ae6-9d9a5ab30af1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:20.782091 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e254431-4f54-4460-8ae6-9d9a5ab30af1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:20.783116 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e254431-4f54-4460-8ae6-9d9a5ab30af1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:30:20.783527 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e254431-4f54-4460-8ae6-9d9a5ab30af1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:20.785359 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e254431-4f54-4460-8ae6-9d9a5ab30af1 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:20.799989 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26cfe3e-f517-4140-b66e-9f3795349ac7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:20.801640 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26cfe3e-f517-4140-b66e-9f3795349ac7 service neutron] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:20.823816 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 862/1741] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:30:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 07:30:20.828308 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 880/1742] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Nov 2 07:30:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:30:20.829529 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26cfe3e-f517-4140-b66e-9f3795349ac7 service neutron] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:20.854923 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26cfe3e-f517-4140-b66e-9f3795349ac7 service neutron] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:20.864404 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1e017d-b512-4566-a036-8f98e0d95b70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:20.865007 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1e017d-b512-4566-a036-8f98e0d95b70 admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:20.865286 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1e017d-b512-4566-a036-8f98e0d95b70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:20.865560 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1e017d-b512-4566-a036-8f98e0d95b70 admin admin] PATH_INFO: `/v3/projects` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:20.866192 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1e017d-b512-4566-a036-8f98e0d95b70 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:20.868569 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1e017d-b512-4566-a036-8f98e0d95b70 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:20.938125 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 863/1743] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Nov 2 07:30:20 2022] POST /identity/v3/projects => generated 369 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:30:20.942646 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e254431-4f54-4460-8ae6-9d9a5ab30af1 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:20.953847 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e254431-4f54-4460-8ae6-9d9a5ab30af1 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:21.038784 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9ab1f6-ebc6-4079-9507-a99d0cedb3c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:21.039695 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9ab1f6-ebc6-4079-9507-a99d0cedb3c0 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:21.040110 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9ab1f6-ebc6-4079-9507-a99d0cedb3c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:21.040508 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9ab1f6-ebc6-4079-9507-a99d0cedb3c0 admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:21.041434 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9ab1f6-ebc6-4079-9507-a99d0cedb3c0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:21.044050 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9ab1f6-ebc6-4079-9507-a99d0cedb3c0 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:21.090449 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 881/1744] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Nov 2 07:30:20 2022] POST /identity/v3/users => generated 285 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 07:30:21.091859 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1e017d-b512-4566-a036-8f98e0d95b70 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:21.115627 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1e017d-b512-4566-a036-8f98e0d95b70 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:21.130599 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef9c224-50e6-4f16-864d-4d26f95ce0b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:21.131481 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef9c224-50e6-4f16-864d-4d26f95ce0b4 admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:21.131954 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef9c224-50e6-4f16-864d-4d26f95ce0b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:21.132377 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef9c224-50e6-4f16-864d-4d26f95ce0b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:21.133259 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef9c224-50e6-4f16-864d-4d26f95ce0b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:21.136662 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef9c224-50e6-4f16-864d-4d26f95ce0b4 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:21.151750 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 864/1745] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:30:21 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 07:30:21.156116 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9ab1f6-ebc6-4079-9507-a99d0cedb3c0 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:21.167700 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9ab1f6-ebc6-4079-9507-a99d0cedb3c0 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:21.173671 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17903ef6-b06b-4b32-8192-97dfbe1e99dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:21.174288 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17903ef6-b06b-4b32-8192-97dfbe1e99dc admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:21.174594 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17903ef6-b06b-4b32-8192-97dfbe1e99dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:21.174935 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17903ef6-b06b-4b32-8192-97dfbe1e99dc admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:21.175755 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17903ef6-b06b-4b32-8192-97dfbe1e99dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:21.177486 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17903ef6-b06b-4b32-8192-97dfbe1e99dc admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:21.205410 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 882/1746] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:30:21 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:30:21.206629 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef9c224-50e6-4f16-864d-4d26f95ce0b4 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:21.221604 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef9c224-50e6-4f16-864d-4d26f95ce0b4 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:21.228800 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be10fce3-3804-496a-ae67-a05f1f7dd620 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:21.230002 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be10fce3-3804-496a-ae67-a05f1f7dd620 admin admin] REQUEST_METHOD: `PUT` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:21.230382 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be10fce3-3804-496a-ae67-a05f1f7dd620 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:21.230742 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be10fce3-3804-496a-ae67-a05f1f7dd620 admin admin] PATH_INFO: `/v3/projects/f0ee3def16ff494e8255614e108b03e4/users/b4d5d3ef367c451f88f9687165b251e9/roles/b494fedd00f24f0b84a529e8672a156a` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:21.255590 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be10fce3-3804-496a-ae67-a05f1f7dd620 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f0ee3def16ff494e8255614e108b03e4, user_id=b4d5d3ef367c451f88f9687165b251e9, role_id=b494fedd00f24f0b84a529e8672a156a)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:21.260172 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be10fce3-3804-496a-ae67-a05f1f7dd620 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:21.281930 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 865/1747] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Nov 2 07:30:21 2022] PUT /identity/v3/projects/f0ee3def16ff494e8255614e108b03e4/users/b4d5d3ef367c451f88f9687165b251e9/roles/b494fedd00f24f0b84a529e8672a156a => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:30:21.292777 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabf3aed-2951-4595-9450-22fadba35bc8 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:21.293065 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabf3aed-2951-4595-9450-22fadba35bc8 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:21.293363 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabf3aed-2951-4595-9450-22fadba35bc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:21.335346 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-cabf3aed-2951-4595-9450-22fadba35bc8 None None] MFA Rules not processed for user `b4d5d3ef367c451f88f9687165b251e9`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:30:21.337993 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-cabf3aed-2951-4595-9450-22fadba35bc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:21.413115 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:30:21.452225 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 883/1748] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:30:21 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:30:21.456954 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be10fce3-3804-496a-ae67-a05f1f7dd620 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:21.560928 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be10fce3-3804-496a-ae67-a05f1f7dd620 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:21.569837 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629d5b64-28f6-4070-b944-25fc8124b587 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:21.570621 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629d5b64-28f6-4070-b944-25fc8124b587 admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:21.571019 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629d5b64-28f6-4070-b944-25fc8124b587 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:21.571385 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629d5b64-28f6-4070-b944-25fc8124b587 admin admin] PATH_INFO: `/v3/projects` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:21.572121 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629d5b64-28f6-4070-b944-25fc8124b587 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:21.581117 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629d5b64-28f6-4070-b944-25fc8124b587 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:21.633483 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabf3aed-2951-4595-9450-22fadba35bc8 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:21.638594 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 866/1749] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Nov 2 07:30:21 2022] POST /identity/v3/projects => generated 367 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:30:21.648552 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabf3aed-2951-4595-9450-22fadba35bc8 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:21.721307 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629d5b64-28f6-4070-b944-25fc8124b587 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:21.730441 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-629d5b64-28f6-4070-b944-25fc8124b587 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:21.834954 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:30:21.865473 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118a42cf-aa92-404e-ab7f-1a98f87013c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:21.866528 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118a42cf-aa92-404e-ab7f-1a98f87013c4 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:21.867124 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118a42cf-aa92-404e-ab7f-1a98f87013c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:21.867556 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118a42cf-aa92-404e-ab7f-1a98f87013c4 admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:21.870808 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118a42cf-aa92-404e-ab7f-1a98f87013c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:21.875155 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118a42cf-aa92-404e-ab7f-1a98f87013c4 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:21.923618 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629d5b64-28f6-4070-b944-25fc8124b587 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:21.946492 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 884/1750] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Nov 2 07:30:21 2022] POST /identity/v3/users => generated 290 bytes in 314 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 07:30:21.950742 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118a42cf-aa92-404e-ab7f-1a98f87013c4 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:21.964395 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8222389-dbde-46a4-a400-e502bd13e6db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:21.968613 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8222389-dbde-46a4-a400-e502bd13e6db admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:21.971376 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8222389-dbde-46a4-a400-e502bd13e6db admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:21.971764 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8222389-dbde-46a4-a400-e502bd13e6db admin admin] PATH_INFO: `/v3/projects/266bef2a1b5843f6aaff57309ecd6551` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:21.983593 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8222389-dbde-46a4-a400-e502bd13e6db admin admin] RBAC: Authorizing `identity:delete_project(project_id=266bef2a1b5843f6aaff57309ecd6551)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:21.996941 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118a42cf-aa92-404e-ab7f-1a98f87013c4 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:21.997161 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8222389-dbde-46a4-a400-e502bd13e6db admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:22.021289 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153053fd-e819-4638-af90-febbc35af279 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:22.024814 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153053fd-e819-4638-af90-febbc35af279 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:22.025215 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153053fd-e819-4638-af90-febbc35af279 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:22.025627 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153053fd-e819-4638-af90-febbc35af279 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:22.026347 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153053fd-e819-4638-af90-febbc35af279 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:22.032996 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153053fd-e819-4638-af90-febbc35af279 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:22.103016 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 885/1751] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:30:21 2022] GET /identity/v3/roles => generated 1420 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 07:30:22.109376 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153053fd-e819-4638-af90-febbc35af279 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:22.181511 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153053fd-e819-4638-af90-febbc35af279 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:22.271958 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-c8222389-dbde-46a4-a400-e502bd13e6db admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '266bef2a1b5843f6aaff57309ecd6551', 'request_id': 'req-c8222389-dbde-46a4-a400-e502bd13e6db'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:22.284152 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-c8222389-dbde-46a4-a400-e502bd13e6db admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '266bef2a1b5843f6aaff57309ecd6551', 'request_id': 'req-c8222389-dbde-46a4-a400-e502bd13e6db'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:22.328083 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-c8222389-dbde-46a4-a400-e502bd13e6db admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '266bef2a1b5843f6aaff57309ecd6551', 'request_id': 'req-c8222389-dbde-46a4-a400-e502bd13e6db'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:22.375517 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-c8222389-dbde-46a4-a400-e502bd13e6db admin admin] The token cache is being invalidate because project 266bef2a1b5843f6aaff57309ecd6551 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81501) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:30:22.377386 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-c8222389-dbde-46a4-a400-e502bd13e6db admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:22.378276 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-c8222389-dbde-46a4-a400-e502bd13e6db admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:22.385854 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 867/1752] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Nov 2 07:30:21 2022] DELETE /identity/v3/projects/266bef2a1b5843f6aaff57309ecd6551 => generated 0 bytes in 663 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:30:22.386898 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fa4c68-3068-456f-9651-c41253fa3505 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:22.387368 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8222389-dbde-46a4-a400-e502bd13e6db admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:22.387865 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fa4c68-3068-456f-9651-c41253fa3505 admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:22.388458 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fa4c68-3068-456f-9651-c41253fa3505 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:22.388931 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fa4c68-3068-456f-9651-c41253fa3505 admin admin] PATH_INFO: `/v3/projects/18d91cff8a324c00b274d4572ae369ab/users/049145763e4543cfae4da0ca6a2ddb41/roles/c4bf019fc6c44c71a6c984f6d0bab75f` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:22.389306 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-c8222389-dbde-46a4-a400-e502bd13e6db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:22.427393 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fa4c68-3068-456f-9651-c41253fa3505 admin admin] RBAC: Authorizing `identity:create_grant(project_id=18d91cff8a324c00b274d4572ae369ab, user_id=049145763e4543cfae4da0ca6a2ddb41, role_id=c4bf019fc6c44c71a6c984f6d0bab75f)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:22.441572 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fa4c68-3068-456f-9651-c41253fa3505 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:22.444991 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8222389-dbde-46a4-a400-e502bd13e6db admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:22.457954 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfc4d66-7421-41e5-9ac3-c8d7bc8baf75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:22.459407 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfc4d66-7421-41e5-9ac3-c8d7bc8baf75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:22.460377 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfc4d66-7421-41e5-9ac3-c8d7bc8baf75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:22.460753 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfc4d66-7421-41e5-9ac3-c8d7bc8baf75 admin admin] PATH_INFO: `/v3/users/59ee6c50550b4254831e31a87ad91c9a` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:22.464008 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfc4d66-7421-41e5-9ac3-c8d7bc8baf75 admin admin] RBAC: Authorizing `identity:delete_user(user_id=59ee6c50550b4254831e31a87ad91c9a)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:22.483762 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfc4d66-7421-41e5-9ac3-c8d7bc8baf75 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:22.491455 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 886/1753] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Nov 2 07:30:22 2022] PUT /identity/v3/projects/18d91cff8a324c00b274d4572ae369ab/users/049145763e4543cfae4da0ca6a2ddb41/roles/c4bf019fc6c44c71a6c984f6d0bab75f => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:30:22.495568 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fa4c68-3068-456f-9651-c41253fa3505 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:22.499653 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-a4fa4c68-3068-456f-9651-c41253fa3505 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:22.613476 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-7dfc4d66-7421-41e5-9ac3-c8d7bc8baf75 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '59ee6c50550b4254831e31a87ad91c9a'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:22.633507 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:30:22.635256 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-7dfc4d66-7421-41e5-9ac3-c8d7bc8baf75 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '59ee6c50550b4254831e31a87ad91c9a'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:22.636237 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-7dfc4d66-7421-41e5-9ac3-c8d7bc8baf75 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '59ee6c50550b4254831e31a87ad91c9a'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:22.648780 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-7dfc4d66-7421-41e5-9ac3-c8d7bc8baf75 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '59ee6c50550b4254831e31a87ad91c9a'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:22.686659 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-7dfc4d66-7421-41e5-9ac3-c8d7bc8baf75 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '59ee6c50550b4254831e31a87ad91c9a'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:22.704988 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 868/1754] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Nov 2 07:30:22 2022] DELETE /identity/v3/users/59ee6c50550b4254831e31a87ad91c9a => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:30:22.711396 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fa4c68-3068-456f-9651-c41253fa3505 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:22.804235 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83684ecb-e645-4524-b52e-344bf450f2d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:22.805527 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83684ecb-e645-4524-b52e-344bf450f2d5 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:22.806021 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83684ecb-e645-4524-b52e-344bf450f2d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:22.806650 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83684ecb-e645-4524-b52e-344bf450f2d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:22.807720 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83684ecb-e645-4524-b52e-344bf450f2d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:22.809756 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83684ecb-e645-4524-b52e-344bf450f2d5 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:22.832010 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfc4d66-7421-41e5-9ac3-c8d7bc8baf75 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:22.832833 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 887/1755] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:30:22 2022] GET /identity/v3/roles => generated 1420 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:30:22.838870 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-7dfc4d66-7421-41e5-9ac3-c8d7bc8baf75 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:22.869276 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfc4d66-7421-41e5-9ac3-c8d7bc8baf75 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:22.898121 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503f8f70-1686-4712-94b8-d146636ed452 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:22.903106 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503f8f70-1686-4712-94b8-d146636ed452 admin admin] REQUEST_METHOD: `PUT` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:22.905424 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503f8f70-1686-4712-94b8-d146636ed452 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:22.906971 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503f8f70-1686-4712-94b8-d146636ed452 admin admin] PATH_INFO: `/v3/projects/18d91cff8a324c00b274d4572ae369ab/users/049145763e4543cfae4da0ca6a2ddb41/roles/c4bf019fc6c44c71a6c984f6d0bab75f` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:22.915972 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503f8f70-1686-4712-94b8-d146636ed452 admin admin] RBAC: Authorizing `identity:create_grant(project_id=18d91cff8a324c00b274d4572ae369ab, user_id=049145763e4543cfae4da0ca6a2ddb41, role_id=c4bf019fc6c44c71a6c984f6d0bab75f)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:22.919199 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503f8f70-1686-4712-94b8-d146636ed452 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:22.962155 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 869/1756] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Nov 2 07:30:22 2022] PUT /identity/v3/projects/18d91cff8a324c00b274d4572ae369ab/users/049145763e4543cfae4da0ca6a2ddb41/roles/c4bf019fc6c44c71a6c984f6d0bab75f => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:30:22.993343 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca6d9bf-721c-4387-bb29-8c87a26256a9 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:22.993628 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca6d9bf-721c-4387-bb29-8c87a26256a9 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:23.017527 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca6d9bf-721c-4387-bb29-8c87a26256a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:23.093968 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-aca6d9bf-721c-4387-bb29-8c87a26256a9 None None] MFA Rules not processed for user `049145763e4543cfae4da0ca6a2ddb41`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:30:23.096077 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-aca6d9bf-721c-4387-bb29-8c87a26256a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:23.512026 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 888/1757] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:30:22 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 533 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:30:23.539472 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503f8f70-1686-4712-94b8-d146636ed452 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:23.541272 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-503f8f70-1686-4712-94b8-d146636ed452 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:23.653715 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=80,keystone:DELETE=17,keystone:INSERT=3 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:30:23.798687 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503f8f70-1686-4712-94b8-d146636ed452 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:23.813188 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41647ef6-df5c-4e43-b341-afb2a62e31b6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43bb326fb82e4d2c83fb2eaf91860dc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:23.813859 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41647ef6-df5c-4e43-b341-afb2a62e31b6 service cinder] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:23.814696 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41647ef6-df5c-4e43-b341-afb2a62e31b6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:23.815593 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41647ef6-df5c-4e43-b341-afb2a62e31b6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:23.846770 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41647ef6-df5c-4e43-b341-afb2a62e31b6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:23.848026 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41647ef6-df5c-4e43-b341-afb2a62e31b6 service cinder] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:23.893335 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 870/1758] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:30:23 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:30:24.078885 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca6d9bf-721c-4387-bb29-8c87a26256a9 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:24.080641 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-aca6d9bf-721c-4387-bb29-8c87a26256a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:24.245781 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca6d9bf-721c-4387-bb29-8c87a26256a9 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:24.260627 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a5e0f2-ecc1-4b71-9235-cc50b397d45d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:24.265127 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a5e0f2-ecc1-4b71-9235-cc50b397d45d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:24.265423 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a5e0f2-ecc1-4b71-9235-cc50b397d45d admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:24.282633 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a5e0f2-ecc1-4b71-9235-cc50b397d45d admin admin] PATH_INFO: `/v3/projects/d8cd884ad482458fa89a9b1fe2847a38` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:24.303429 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a5e0f2-ecc1-4b71-9235-cc50b397d45d admin admin] RBAC: Authorizing `identity:delete_project(project_id=d8cd884ad482458fa89a9b1fe2847a38)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:24.317966 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a5e0f2-ecc1-4b71-9235-cc50b397d45d admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:24.416308 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:30:24.484799 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-61a5e0f2-ecc1-4b71-9235-cc50b397d45d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd8cd884ad482458fa89a9b1fe2847a38', 'request_id': 'req-61a5e0f2-ecc1-4b71-9235-cc50b397d45d'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:24.509309 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-61a5e0f2-ecc1-4b71-9235-cc50b397d45d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd8cd884ad482458fa89a9b1fe2847a38', 'request_id': 'req-61a5e0f2-ecc1-4b71-9235-cc50b397d45d'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:24.524267 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-61a5e0f2-ecc1-4b71-9235-cc50b397d45d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd8cd884ad482458fa89a9b1fe2847a38', 'request_id': 'req-61a5e0f2-ecc1-4b71-9235-cc50b397d45d'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:24.555765 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-61a5e0f2-ecc1-4b71-9235-cc50b397d45d admin admin] The token cache is being invalidate because project d8cd884ad482458fa89a9b1fe2847a38 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:30:24.556120 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-61a5e0f2-ecc1-4b71-9235-cc50b397d45d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:24.564883 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-61a5e0f2-ecc1-4b71-9235-cc50b397d45d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:24.567875 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 889/1759] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Nov 2 07:30:24 2022] DELETE /identity/v3/projects/d8cd884ad482458fa89a9b1fe2847a38 => generated 0 bytes in 490 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:30:24.572499 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41647ef6-df5c-4e43-b341-afb2a62e31b6 service cinder] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:24.612889 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41647ef6-df5c-4e43-b341-afb2a62e31b6 service cinder] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:24.820797 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b31217b-57b5-419b-85e4-71de224b4a1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:24.821684 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b31217b-57b5-419b-85e4-71de224b4a1f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:24.822601 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b31217b-57b5-419b-85e4-71de224b4a1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:24.823016 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b31217b-57b5-419b-85e4-71de224b4a1f admin admin] PATH_INFO: `/v3/users/eb2cec8baf6a482eac2e1d5422f18377` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:24.833760 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b31217b-57b5-419b-85e4-71de224b4a1f admin admin] RBAC: Authorizing `identity:delete_user(user_id=eb2cec8baf6a482eac2e1d5422f18377)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:24.835898 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b31217b-57b5-419b-85e4-71de224b4a1f admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:24.966693 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-5b31217b-57b5-419b-85e4-71de224b4a1f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eb2cec8baf6a482eac2e1d5422f18377'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:24.985205 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-5b31217b-57b5-419b-85e4-71de224b4a1f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eb2cec8baf6a482eac2e1d5422f18377'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:24.985898 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-5b31217b-57b5-419b-85e4-71de224b4a1f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eb2cec8baf6a482eac2e1d5422f18377'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:25.030215 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-5b31217b-57b5-419b-85e4-71de224b4a1f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eb2cec8baf6a482eac2e1d5422f18377'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:25.062177 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-5b31217b-57b5-419b-85e4-71de224b4a1f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eb2cec8baf6a482eac2e1d5422f18377'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:25.075554 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 871/1760] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Nov 2 07:30:24 2022] DELETE /identity/v3/users/eb2cec8baf6a482eac2e1d5422f18377 => generated 0 bytes in 503 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:30:25.944424 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a5e0f2-ecc1-4b71-9235-cc50b397d45d admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:25.946976 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-61a5e0f2-ecc1-4b71-9235-cc50b397d45d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:25.967507 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a5e0f2-ecc1-4b71-9235-cc50b397d45d admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:26.085042 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bbc65c-bec8-41bc-ad37-c77ebf7d5787 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:26.086134 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bbc65c-bec8-41bc-ad37-c77ebf7d5787 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:26.086909 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bbc65c-bec8-41bc-ad37-c77ebf7d5787 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:26.087346 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bbc65c-bec8-41bc-ad37-c77ebf7d5787 admin admin] PATH_INFO: `/v3/projects/d6094e5ca08540afbe75fa5e0fef340a` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:26.089042 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bbc65c-bec8-41bc-ad37-c77ebf7d5787 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d6094e5ca08540afbe75fa5e0fef340a)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:26.101264 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bbc65c-bec8-41bc-ad37-c77ebf7d5787 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:26.217075 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-b9bbc65c-bec8-41bc-ad37-c77ebf7d5787 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd6094e5ca08540afbe75fa5e0fef340a', 'request_id': 'req-b9bbc65c-bec8-41bc-ad37-c77ebf7d5787'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:26.231873 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-b9bbc65c-bec8-41bc-ad37-c77ebf7d5787 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd6094e5ca08540afbe75fa5e0fef340a', 'request_id': 'req-b9bbc65c-bec8-41bc-ad37-c77ebf7d5787'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:26.250481 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-b9bbc65c-bec8-41bc-ad37-c77ebf7d5787 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd6094e5ca08540afbe75fa5e0fef340a', 'request_id': 'req-b9bbc65c-bec8-41bc-ad37-c77ebf7d5787'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:26.290091 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-b9bbc65c-bec8-41bc-ad37-c77ebf7d5787 admin admin] The token cache is being invalidate because project d6094e5ca08540afbe75fa5e0fef340a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:30:26.290091 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-b9bbc65c-bec8-41bc-ad37-c77ebf7d5787 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:26.290091 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-b9bbc65c-bec8-41bc-ad37-c77ebf7d5787 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:26.295108 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 890/1761] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Nov 2 07:30:25 2022] DELETE /identity/v3/projects/d6094e5ca08540afbe75fa5e0fef340a => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:30:26.297143 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b66b5dd-cff4-4e15-9c73-976b00ff359c None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:26.297607 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b66b5dd-cff4-4e15-9c73-976b00ff359c None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:26.297862 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b66b5dd-cff4-4e15-9c73-976b00ff359c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:26.332169 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-9b66b5dd-cff4-4e15-9c73-976b00ff359c None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:30:26.333409 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-9b66b5dd-cff4-4e15-9c73-976b00ff359c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:26.419539 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:30:26.468459 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 872/1762] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:30:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:30:26.476348 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f59026-58b6-4e1e-b2dc-b5e689ab60ed None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:26.476626 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f59026-58b6-4e1e-b2dc-b5e689ab60ed None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:26.476897 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f59026-58b6-4e1e-b2dc-b5e689ab60ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:26.535968 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-92f59026-58b6-4e1e-b2dc-b5e689ab60ed None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:30:26.538660 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-92f59026-58b6-4e1e-b2dc-b5e689ab60ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:26.556553 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 891/1763] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:30:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:30:26.560767 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b66b5dd-cff4-4e15-9c73-976b00ff359c None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:26.571874 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b66b5dd-cff4-4e15-9c73-976b00ff359c None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:26.578112 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb5f610-9c6e-4a18-8cb0-2eff5c11780e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:26.578957 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb5f610-9c6e-4a18-8cb0-2eff5c11780e admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:26.579375 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb5f610-9c6e-4a18-8cb0-2eff5c11780e admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:26.579800 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb5f610-9c6e-4a18-8cb0-2eff5c11780e admin admin] PATH_INFO: `/v3/domains` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:26.580582 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb5f610-9c6e-4a18-8cb0-2eff5c11780e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:30:26.581056 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb5f610-9c6e-4a18-8cb0-2eff5c11780e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:26.583655 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb5f610-9c6e-4a18-8cb0-2eff5c11780e admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:26.609781 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f59026-58b6-4e1e-b2dc-b5e689ab60ed None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:26.626549 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 873/1764] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Nov 2 07:30:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:30:26.631008 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f59026-58b6-4e1e-b2dc-b5e689ab60ed None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:26.653008 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55926443-b99b-4bf4-b172-e371203a5824 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:26.653640 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55926443-b99b-4bf4-b172-e371203a5824 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:26.658945 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55926443-b99b-4bf4-b172-e371203a5824 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:26.659243 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55926443-b99b-4bf4-b172-e371203a5824 admin admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:26.659916 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55926443-b99b-4bf4-b172-e371203a5824 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:26.665627 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55926443-b99b-4bf4-b172-e371203a5824 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:26.713852 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 892/1765] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Nov 2 07:30:26 2022] POST /identity/v3/projects => generated 373 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:30:26.718790 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb5f610-9c6e-4a18-8cb0-2eff5c11780e admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:26.738959 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb5f610-9c6e-4a18-8cb0-2eff5c11780e admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:26.809321 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55926443-b99b-4bf4-b172-e371203a5824 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:26.812006 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-55926443-b99b-4bf4-b172-e371203a5824 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:26.828805 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:30:26.872902 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83d3d6e-5c2d-48f8-8d44-51239166f3fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:26.873823 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83d3d6e-5c2d-48f8-8d44-51239166f3fb admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:26.874108 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83d3d6e-5c2d-48f8-8d44-51239166f3fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:26.874462 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83d3d6e-5c2d-48f8-8d44-51239166f3fb admin admin] PATH_INFO: `/v3/users` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:26.875101 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83d3d6e-5c2d-48f8-8d44-51239166f3fb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:26.880273 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83d3d6e-5c2d-48f8-8d44-51239166f3fb admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:26.918616 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 874/1766] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Nov 2 07:30:26 2022] POST /identity/v3/users => generated 287 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:30:26.922391 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83d3d6e-5c2d-48f8-8d44-51239166f3fb admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:26.932981 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83d3d6e-5c2d-48f8-8d44-51239166f3fb admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:26.936594 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55926443-b99b-4bf4-b172-e371203a5824 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:26.939401 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3958c34f-4d41-4fba-b24f-604b34c9d396 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:26.940084 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3958c34f-4d41-4fba-b24f-604b34c9d396 admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:26.940426 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3958c34f-4d41-4fba-b24f-604b34c9d396 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:26.940811 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3958c34f-4d41-4fba-b24f-604b34c9d396 admin admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:26.941897 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3958c34f-4d41-4fba-b24f-604b34c9d396 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:26.942606 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09258a3f-e651-41b8-9477-661edbdba065 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:26.943290 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3958c34f-4d41-4fba-b24f-604b34c9d396 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:26.943397 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09258a3f-e651-41b8-9477-661edbdba065 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:26.943634 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09258a3f-e651-41b8-9477-661edbdba065 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:26.943966 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09258a3f-e651-41b8-9477-661edbdba065 admin admin] PATH_INFO: `/v3/users/b4d5d3ef367c451f88f9687165b251e9` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:26.945189 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09258a3f-e651-41b8-9477-661edbdba065 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b4d5d3ef367c451f88f9687165b251e9)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:26.947206 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09258a3f-e651-41b8-9477-661edbdba065 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:26.956294 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 875/1767] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:30:26 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 07:30:26.963754 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3958c34f-4d41-4fba-b24f-604b34c9d396 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:26.980887 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3958c34f-4d41-4fba-b24f-604b34c9d396 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:27.000305 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e028bbb-35fc-4c11-a627-a665deb31ccf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:27.003701 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e028bbb-35fc-4c11-a627-a665deb31ccf admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:27.003789 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e028bbb-35fc-4c11-a627-a665deb31ccf admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:27.004063 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e028bbb-35fc-4c11-a627-a665deb31ccf admin admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:27.004722 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e028bbb-35fc-4c11-a627-a665deb31ccf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:27.009553 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e028bbb-35fc-4c11-a627-a665deb31ccf admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:27.019460 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-09258a3f-e651-41b8-9477-661edbdba065 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b4d5d3ef367c451f88f9687165b251e9'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:27.046716 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-09258a3f-e651-41b8-9477-661edbdba065 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b4d5d3ef367c451f88f9687165b251e9'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:27.048229 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-09258a3f-e651-41b8-9477-661edbdba065 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b4d5d3ef367c451f88f9687165b251e9'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:27.062510 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-09258a3f-e651-41b8-9477-661edbdba065 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b4d5d3ef367c451f88f9687165b251e9'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:27.064489 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 876/1768] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:30:26 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:30:27.066851 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-09258a3f-e651-41b8-9477-661edbdba065 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b4d5d3ef367c451f88f9687165b251e9'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:27.069939 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e028bbb-35fc-4c11-a627-a665deb31ccf admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:27.075578 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-3e028bbb-35fc-4c11-a627-a665deb31ccf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:27.112593 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e028bbb-35fc-4c11-a627-a665deb31ccf admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:27.127937 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c25df4-ea07-43ae-92f5-100b788f3c33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:27.129821 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 893/1769] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Nov 2 07:30:26 2022] DELETE /identity/v3/users/b4d5d3ef367c451f88f9687165b251e9 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:30:27.136007 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a42751-ba6a-4a81-96f6-e27b4dddcf0f None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:27.136377 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a42751-ba6a-4a81-96f6-e27b4dddcf0f None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:27.136725 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a42751-ba6a-4a81-96f6-e27b4dddcf0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:27.139327 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c25df4-ea07-43ae-92f5-100b788f3c33 admin admin] REQUEST_METHOD: `PUT` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:27.139648 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c25df4-ea07-43ae-92f5-100b788f3c33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:27.140020 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c25df4-ea07-43ae-92f5-100b788f3c33 admin admin] PATH_INFO: `/v3/projects/71f746654bee4209b28de275a9ce5116/users/db5d8789175745d398d0011f616785c9/roles/b494fedd00f24f0b84a529e8672a156a` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:27.163262 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-40a42751-ba6a-4a81-96f6-e27b4dddcf0f None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:30:27.164406 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c25df4-ea07-43ae-92f5-100b788f3c33 admin admin] RBAC: Authorizing `identity:create_grant(project_id=71f746654bee4209b28de275a9ce5116, user_id=db5d8789175745d398d0011f616785c9, role_id=b494fedd00f24f0b84a529e8672a156a)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:27.167212 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c25df4-ea07-43ae-92f5-100b788f3c33 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:27.172098 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-40a42751-ba6a-4a81-96f6-e27b4dddcf0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:27.192261 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 877/1770] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Nov 2 07:30:27 2022] PUT /identity/v3/projects/71f746654bee4209b28de275a9ce5116/users/db5d8789175745d398d0011f616785c9/roles/b494fedd00f24f0b84a529e8672a156a => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:30:27.202578 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d601949-fd3a-452a-b518-856ed5f36ce7 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:27.202811 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d601949-fd3a-452a-b518-856ed5f36ce7 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:27.203085 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d601949-fd3a-452a-b518-856ed5f36ce7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:27.240945 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-9d601949-fd3a-452a-b518-856ed5f36ce7 None None] MFA Rules not processed for user `db5d8789175745d398d0011f616785c9`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:30:27.243026 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-9d601949-fd3a-452a-b518-856ed5f36ce7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:27.265288 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 894/1771] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:30:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:30:27.275727 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a42751-ba6a-4a81-96f6-e27b4dddcf0f None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:27.277437 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-40a42751-ba6a-4a81-96f6-e27b4dddcf0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:27.316879 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:30:27.342445 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 878/1772] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:30:27 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:30:27.355051 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d601949-fd3a-452a-b518-856ed5f36ce7 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:27.356478 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a42751-ba6a-4a81-96f6-e27b4dddcf0f None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:27.356808 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-9d601949-fd3a-452a-b518-856ed5f36ce7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:27.366674 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:30:27.438690 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1741fef-e254-4c35-8491-5190cbf663b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3ec6ecb85f94ef59762877d9ea33ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:27.440000 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1741fef-e254-4c35-8491-5190cbf663b7 service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:27.440441 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1741fef-e254-4c35-8491-5190cbf663b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:27.440725 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1741fef-e254-4c35-8491-5190cbf663b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:27.458303 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d601949-fd3a-452a-b518-856ed5f36ce7 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:27.462636 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382a8fbc-2f1c-45ae-9203-1613ad00bc0c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43bb326fb82e4d2c83fb2eaf91860dc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:27.463256 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382a8fbc-2f1c-45ae-9203-1613ad00bc0c service cinder] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:27.463534 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382a8fbc-2f1c-45ae-9203-1613ad00bc0c service cinder] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:27.463804 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382a8fbc-2f1c-45ae-9203-1613ad00bc0c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:27.474080 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382a8fbc-2f1c-45ae-9203-1613ad00bc0c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:27.475560 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382a8fbc-2f1c-45ae-9203-1613ad00bc0c service cinder] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:27.496220 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 879/1773] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:30:27 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 07:30:27.536272 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1741fef-e254-4c35-8491-5190cbf663b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:27.538136 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1741fef-e254-4c35-8491-5190cbf663b7 service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:27.569143 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 895/1774] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:30:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 07:30:27.917223 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e31c8d5-f60a-4b9f-bc69-c7f141e9b9ab None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:27.917512 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e31c8d5-f60a-4b9f-bc69-c7f141e9b9ab None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:27.917785 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e31c8d5-f60a-4b9f-bc69-c7f141e9b9ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:27.984461 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-3e31c8d5-f60a-4b9f-bc69-c7f141e9b9ab None None] MFA Rules not processed for user `fe26aea6bb9d4dcebf046da719b2c244`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:30:27.986317 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-3e31c8d5-f60a-4b9f-bc69-c7f141e9b9ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:28.147322 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 880/1775] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:30:27 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:30:28.152248 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1741fef-e254-4c35-8491-5190cbf663b7 service neutron] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:28.153756 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-d1741fef-e254-4c35-8491-5190cbf663b7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:28.166825 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1741fef-e254-4c35-8491-5190cbf663b7 service neutron] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:28.171502 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f960cc8f-771c-4b82-b532-9f16ad437dd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:28.172106 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f960cc8f-771c-4b82-b532-9f16ad437dd8 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:28.172384 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f960cc8f-771c-4b82-b532-9f16ad437dd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:28.172661 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f960cc8f-771c-4b82-b532-9f16ad437dd8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:28.173290 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f960cc8f-771c-4b82-b532-9f16ad437dd8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:28.175242 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f960cc8f-771c-4b82-b532-9f16ad437dd8 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:28.212461 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 896/1776] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Nov 2 07:30:28 2022] POST /identity/v3/projects => generated 379 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 07:30:28.217537 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e31c8d5-f60a-4b9f-bc69-c7f141e9b9ab None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:28.219051 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-3e31c8d5-f60a-4b9f-bc69-c7f141e9b9ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:28.222040 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f960cc8f-771c-4b82-b532-9f16ad437dd8 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:28.233736 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e31c8d5-f60a-4b9f-bc69-c7f141e9b9ab None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:28.295862 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c12377-9c65-4178-80bd-91bd57b1a326 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:28.296516 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c12377-9c65-4178-80bd-91bd57b1a326 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:28.296772 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c12377-9c65-4178-80bd-91bd57b1a326 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:28.297039 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c12377-9c65-4178-80bd-91bd57b1a326 admin admin] PATH_INFO: `/v3/projects/f0ee3def16ff494e8255614e108b03e4` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:28.298130 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c12377-9c65-4178-80bd-91bd57b1a326 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f0ee3def16ff494e8255614e108b03e4)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:28.300279 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c12377-9c65-4178-80bd-91bd57b1a326 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:28.309910 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f960cc8f-771c-4b82-b532-9f16ad437dd8 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:28.315294 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae0f747-0b6b-49d9-b750-7005bf48c95a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:28.315937 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae0f747-0b6b-49d9-b750-7005bf48c95a admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:28.316205 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae0f747-0b6b-49d9-b750-7005bf48c95a admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:28.316470 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae0f747-0b6b-49d9-b750-7005bf48c95a admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:28.317185 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae0f747-0b6b-49d9-b750-7005bf48c95a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:28.319035 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae0f747-0b6b-49d9-b750-7005bf48c95a admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:28.336691 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:30:28.345204 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 897/1777] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Nov 2 07:30:28 2022] POST /identity/v3/users => generated 296 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:30:28.345347 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:30:28.346614 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae0f747-0b6b-49d9-b750-7005bf48c95a admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:28.353649 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-50c12377-9c65-4178-80bd-91bd57b1a326 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f0ee3def16ff494e8255614e108b03e4', 'request_id': 'req-50c12377-9c65-4178-80bd-91bd57b1a326'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:28.358294 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae0f747-0b6b-49d9-b750-7005bf48c95a admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:28.362259 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-50c12377-9c65-4178-80bd-91bd57b1a326 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f0ee3def16ff494e8255614e108b03e4', 'request_id': 'req-50c12377-9c65-4178-80bd-91bd57b1a326'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:28.371638 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-50c12377-9c65-4178-80bd-91bd57b1a326 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f0ee3def16ff494e8255614e108b03e4', 'request_id': 'req-50c12377-9c65-4178-80bd-91bd57b1a326'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:28.383064 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-50c12377-9c65-4178-80bd-91bd57b1a326 admin admin] The token cache is being invalidate because project f0ee3def16ff494e8255614e108b03e4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81501) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:30:28.383350 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-50c12377-9c65-4178-80bd-91bd57b1a326 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:28.384010 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-50c12377-9c65-4178-80bd-91bd57b1a326 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:28.385917 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 881/1778] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Nov 2 07:30:28 2022] DELETE /identity/v3/projects/f0ee3def16ff494e8255614e108b03e4 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:30:28.390626 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c12377-9c65-4178-80bd-91bd57b1a326 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:28.394587 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-50c12377-9c65-4178-80bd-91bd57b1a326 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:28.428232 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f400e9-7c5d-47d3-b972-5d9c454103f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:28.432101 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f400e9-7c5d-47d3-b972-5d9c454103f5 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:28.432101 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f400e9-7c5d-47d3-b972-5d9c454103f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:28.432101 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f400e9-7c5d-47d3-b972-5d9c454103f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:28.432101 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f400e9-7c5d-47d3-b972-5d9c454103f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:28.433449 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f400e9-7c5d-47d3-b972-5d9c454103f5 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:28.450405 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 898/1779] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:30:28 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:30:28.453512 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f400e9-7c5d-47d3-b972-5d9c454103f5 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:28.455408 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-a9f400e9-7c5d-47d3-b972-5d9c454103f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:28.471165 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f400e9-7c5d-47d3-b972-5d9c454103f5 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:28.475892 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd293cd-957e-472c-a95d-636fe186b87b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:28.476534 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c12377-9c65-4178-80bd-91bd57b1a326 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:28.476661 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd293cd-957e-472c-a95d-636fe186b87b admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:28.476882 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd293cd-957e-472c-a95d-636fe186b87b admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:28.477156 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd293cd-957e-472c-a95d-636fe186b87b admin admin] PATH_INFO: `/v3/projects/905677f4e8c14a66b988145f6913fca6/users/905e2a47c1e34de394b4b380521d364e/roles/c4bf019fc6c44c71a6c984f6d0bab75f` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:28.481560 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1f3a9a-4c39-4c2f-ab08-aa419bc7d393 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:28.482465 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1f3a9a-4c39-4c2f-ab08-aa419bc7d393 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:28.482743 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1f3a9a-4c39-4c2f-ab08-aa419bc7d393 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:28.483004 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1f3a9a-4c39-4c2f-ab08-aa419bc7d393 admin admin] PATH_INFO: `/v3/users/049145763e4543cfae4da0ca6a2ddb41` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:28.484141 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1f3a9a-4c39-4c2f-ab08-aa419bc7d393 admin admin] RBAC: Authorizing `identity:delete_user(user_id=049145763e4543cfae4da0ca6a2ddb41)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:28.486199 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1f3a9a-4c39-4c2f-ab08-aa419bc7d393 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:28.493865 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd293cd-957e-472c-a95d-636fe186b87b admin admin] RBAC: Authorizing `identity:create_grant(project_id=905677f4e8c14a66b988145f6913fca6, user_id=905e2a47c1e34de394b4b380521d364e, role_id=c4bf019fc6c44c71a6c984f6d0bab75f)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:28.496695 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd293cd-957e-472c-a95d-636fe186b87b admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:28.515624 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 899/1780] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Nov 2 07:30:28 2022] PUT /identity/v3/projects/905677f4e8c14a66b988145f6913fca6/users/905e2a47c1e34de394b4b380521d364e/roles/c4bf019fc6c44c71a6c984f6d0bab75f => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:30:28.519392 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd293cd-957e-472c-a95d-636fe186b87b admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:28.546392 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-5b1f3a9a-4c39-4c2f-ab08-aa419bc7d393 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '049145763e4543cfae4da0ca6a2ddb41'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:28.554673 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-5b1f3a9a-4c39-4c2f-ab08-aa419bc7d393 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '049145763e4543cfae4da0ca6a2ddb41'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:28.555080 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-5b1f3a9a-4c39-4c2f-ab08-aa419bc7d393 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '049145763e4543cfae4da0ca6a2ddb41'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:28.565430 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-5b1f3a9a-4c39-4c2f-ab08-aa419bc7d393 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '049145763e4543cfae4da0ca6a2ddb41'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:28.584961 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-5b1f3a9a-4c39-4c2f-ab08-aa419bc7d393 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '049145763e4543cfae4da0ca6a2ddb41'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:28.592842 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 882/1781] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Nov 2 07:30:28 2022] DELETE /identity/v3/users/049145763e4543cfae4da0ca6a2ddb41 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 07:30:28.597886 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd293cd-957e-472c-a95d-636fe186b87b admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:28.678656 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f28de2d-3e48-4593-81aa-d6e3b7520c7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:28.679040 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f28de2d-3e48-4593-81aa-d6e3b7520c7a admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:28.679375 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f28de2d-3e48-4593-81aa-d6e3b7520c7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:28.679655 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f28de2d-3e48-4593-81aa-d6e3b7520c7a admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:28.680300 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f28de2d-3e48-4593-81aa-d6e3b7520c7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:28.682049 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f28de2d-3e48-4593-81aa-d6e3b7520c7a admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:28.697588 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 900/1782] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:30:28 2022] GET /identity/v3/roles => generated 1420 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 07:30:28.702038 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1f3a9a-4c39-4c2f-ab08-aa419bc7d393 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:28.703906 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-5b1f3a9a-4c39-4c2f-ab08-aa419bc7d393 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:28.723374 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1f3a9a-4c39-4c2f-ab08-aa419bc7d393 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:28.729422 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34198fac-d510-43a7-a423-332c698709e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:28.730666 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34198fac-d510-43a7-a423-332c698709e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:28.731325 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34198fac-d510-43a7-a423-332c698709e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:28.731510 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34198fac-d510-43a7-a423-332c698709e0 admin admin] PATH_INFO: `/v3/projects/905677f4e8c14a66b988145f6913fca6/users/905e2a47c1e34de394b4b380521d364e/roles/c4bf019fc6c44c71a6c984f6d0bab75f` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:28.736000 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34198fac-d510-43a7-a423-332c698709e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=905677f4e8c14a66b988145f6913fca6, user_id=905e2a47c1e34de394b4b380521d364e, role_id=c4bf019fc6c44c71a6c984f6d0bab75f)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:28.739683 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34198fac-d510-43a7-a423-332c698709e0 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:28.759083 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 883/1783] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Nov 2 07:30:28 2022] PUT /identity/v3/projects/905677f4e8c14a66b988145f6913fca6/users/905e2a47c1e34de394b4b380521d364e/roles/c4bf019fc6c44c71a6c984f6d0bab75f => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:30:29.176188 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6f2da9-60d2-4691-a2ae-0cd8f23eb4d7 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:29.176622 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6f2da9-60d2-4691-a2ae-0cd8f23eb4d7 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:29.176782 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6f2da9-60d2-4691-a2ae-0cd8f23eb4d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:29.262088 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-af6f2da9-60d2-4691-a2ae-0cd8f23eb4d7 None None] MFA Rules not processed for user `db5d8789175745d398d0011f616785c9`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:30:29.264379 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-af6f2da9-60d2-4691-a2ae-0cd8f23eb4d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:29.287853 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:30:29.386415 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 901/1784] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:30:29 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:30:29.975397 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34198fac-d510-43a7-a423-332c698709e0 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:29.978484 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-34198fac-d510-43a7-a423-332c698709e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:30.064459 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:30:30.121887 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34198fac-d510-43a7-a423-332c698709e0 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:30.128771 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8179627-a156-4d68-8f9e-d1ee1035e582 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:30.129498 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8179627-a156-4d68-8f9e-d1ee1035e582 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:30.129845 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8179627-a156-4d68-8f9e-d1ee1035e582 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:30.130181 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8179627-a156-4d68-8f9e-d1ee1035e582 admin admin] PATH_INFO: `/v3/projects/18d91cff8a324c00b274d4572ae369ab` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:30.131498 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8179627-a156-4d68-8f9e-d1ee1035e582 admin admin] RBAC: Authorizing `identity:delete_project(project_id=18d91cff8a324c00b274d4572ae369ab)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:30.133722 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8179627-a156-4d68-8f9e-d1ee1035e582 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:30.215878 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-d8179627-a156-4d68-8f9e-d1ee1035e582 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '18d91cff8a324c00b274d4572ae369ab', 'request_id': 'req-d8179627-a156-4d68-8f9e-d1ee1035e582'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:30.231256 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-d8179627-a156-4d68-8f9e-d1ee1035e582 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '18d91cff8a324c00b274d4572ae369ab', 'request_id': 'req-d8179627-a156-4d68-8f9e-d1ee1035e582'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:30.245688 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-d8179627-a156-4d68-8f9e-d1ee1035e582 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '18d91cff8a324c00b274d4572ae369ab', 'request_id': 'req-d8179627-a156-4d68-8f9e-d1ee1035e582'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:30.272806 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-d8179627-a156-4d68-8f9e-d1ee1035e582 admin admin] The token cache is being invalidate because project 18d91cff8a324c00b274d4572ae369ab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81501) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:30:30.273161 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-d8179627-a156-4d68-8f9e-d1ee1035e582 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:30.273834 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-d8179627-a156-4d68-8f9e-d1ee1035e582 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:30.280591 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 884/1785] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Nov 2 07:30:29 2022] DELETE /identity/v3/projects/18d91cff8a324c00b274d4572ae369ab => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:30:30.283007 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ce8b6a-8656-4187-90c7-5f58cb6fc35a None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:30.283007 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ce8b6a-8656-4187-90c7-5f58cb6fc35a None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:30.283007 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ce8b6a-8656-4187-90c7-5f58cb6fc35a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:30.317858 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-89ce8b6a-8656-4187-90c7-5f58cb6fc35a None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:30:30.319178 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-89ce8b6a-8656-4187-90c7-5f58cb6fc35a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:30.469729 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 902/1786] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:30:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:30:30.477814 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42540a98-4ced-43e3-8298-334011425d56 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:30.478108 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42540a98-4ced-43e3-8298-334011425d56 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:30.478404 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42540a98-4ced-43e3-8298-334011425d56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:30.504099 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-42540a98-4ced-43e3-8298-334011425d56 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:30:30.505406 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-42540a98-4ced-43e3-8298-334011425d56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:30.518422 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 885/1787] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:30:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:30:30.523687 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ce8b6a-8656-4187-90c7-5f58cb6fc35a None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:30.542600 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ce8b6a-8656-4187-90c7-5f58cb6fc35a None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:30.571137 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58d1590-7466-457a-a068-8ce2462b9b2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:30.571137 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58d1590-7466-457a-a068-8ce2462b9b2f admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:30.571137 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58d1590-7466-457a-a068-8ce2462b9b2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:30.571137 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58d1590-7466-457a-a068-8ce2462b9b2f admin admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:30.571614 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58d1590-7466-457a-a068-8ce2462b9b2f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:30:30.571614 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58d1590-7466-457a-a068-8ce2462b9b2f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:30.584621 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58d1590-7466-457a-a068-8ce2462b9b2f admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:30.676520 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 903/1788] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Nov 2 07:30:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:30:30.681332 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42540a98-4ced-43e3-8298-334011425d56 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:30.696153 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42540a98-4ced-43e3-8298-334011425d56 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:30.706297 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a725bc-c27a-42ff-a993-42420ad9dce3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:30.706955 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a725bc-c27a-42ff-a993-42420ad9dce3 admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:30.707240 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a725bc-c27a-42ff-a993-42420ad9dce3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:30.707512 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a725bc-c27a-42ff-a993-42420ad9dce3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:30.708153 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a725bc-c27a-42ff-a993-42420ad9dce3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:30.710251 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a725bc-c27a-42ff-a993-42420ad9dce3 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:30.798328 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 886/1789] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Nov 2 07:30:30 2022] POST /identity/v3/projects => generated 385 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:30:30.799616 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58d1590-7466-457a-a068-8ce2462b9b2f admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:30.812953 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58d1590-7466-457a-a068-8ce2462b9b2f admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:30.941518 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c285d2-8687-43bf-adb0-153b5edf9dd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:30.942149 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c285d2-8687-43bf-adb0-153b5edf9dd3 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:30.942934 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c285d2-8687-43bf-adb0-153b5edf9dd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:30.942934 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c285d2-8687-43bf-adb0-153b5edf9dd3 admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:30.943354 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c285d2-8687-43bf-adb0-153b5edf9dd3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:30.946519 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c285d2-8687-43bf-adb0-153b5edf9dd3 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:30.996329 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 904/1790] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Nov 2 07:30:30 2022] POST /identity/v3/users => generated 293 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:30:31.000496 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a725bc-c27a-42ff-a993-42420ad9dce3 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:31.012911 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a725bc-c27a-42ff-a993-42420ad9dce3 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:31.043208 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61d6db2-04a5-4082-963a-d1e12a613fd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:31.046764 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61d6db2-04a5-4082-963a-d1e12a613fd7 admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:31.047328 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61d6db2-04a5-4082-963a-d1e12a613fd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:31.048837 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61d6db2-04a5-4082-963a-d1e12a613fd7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:31.049813 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61d6db2-04a5-4082-963a-d1e12a613fd7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:31.058262 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61d6db2-04a5-4082-963a-d1e12a613fd7 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:31.106532 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 887/1791] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:30:30 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:30:31.110759 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c285d2-8687-43bf-adb0-153b5edf9dd3 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:31.125067 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c285d2-8687-43bf-adb0-153b5edf9dd3 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:31.145737 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21574ae1-e438-47e1-bb34-bc352e848787 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:31.146620 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21574ae1-e438-47e1-bb34-bc352e848787 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:31.147079 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21574ae1-e438-47e1-bb34-bc352e848787 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:31.147509 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21574ae1-e438-47e1-bb34-bc352e848787 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:31.148386 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21574ae1-e438-47e1-bb34-bc352e848787 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:31.152036 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21574ae1-e438-47e1-bb34-bc352e848787 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:31.186769 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 905/1792] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:30:31 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:30:31.188324 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61d6db2-04a5-4082-963a-d1e12a613fd7 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:31.209692 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61d6db2-04a5-4082-963a-d1e12a613fd7 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:31.218065 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0614cabe-70cf-41cb-841b-a3e1302e9ea2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:31.219113 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0614cabe-70cf-41cb-841b-a3e1302e9ea2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:31.219535 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0614cabe-70cf-41cb-841b-a3e1302e9ea2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:31.220229 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0614cabe-70cf-41cb-841b-a3e1302e9ea2 admin admin] PATH_INFO: `/v3/projects/90dfa7f098194bd8bada90ca84f058ca/users/7750009aa9f244cc8ff4ac732eddde5a/roles/b494fedd00f24f0b84a529e8672a156a` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:31.241893 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0614cabe-70cf-41cb-841b-a3e1302e9ea2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=90dfa7f098194bd8bada90ca84f058ca, user_id=7750009aa9f244cc8ff4ac732eddde5a, role_id=b494fedd00f24f0b84a529e8672a156a)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:31.246725 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0614cabe-70cf-41cb-841b-a3e1302e9ea2 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:31.292836 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 888/1793] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Nov 2 07:30:31 2022] PUT /identity/v3/projects/90dfa7f098194bd8bada90ca84f058ca/users/7750009aa9f244cc8ff4ac732eddde5a/roles/b494fedd00f24f0b84a529e8672a156a => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:30:31.294231 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eafddde-555d-4217-8e9e-865edba7f27f None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:31.295934 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eafddde-555d-4217-8e9e-865edba7f27f None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:31.296327 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eafddde-555d-4217-8e9e-865edba7f27f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:31.374953 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-4eafddde-555d-4217-8e9e-865edba7f27f None None] MFA Rules not processed for user `7750009aa9f244cc8ff4ac732eddde5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:30:31.379149 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-4eafddde-555d-4217-8e9e-865edba7f27f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:31.424727 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:30:31.574283 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 906/1794] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:30:31 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:30:31.589113 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0614cabe-70cf-41cb-841b-a3e1302e9ea2 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:31.592525 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-0614cabe-70cf-41cb-841b-a3e1302e9ea2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:31.745149 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0614cabe-70cf-41cb-841b-a3e1302e9ea2 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:31.749623 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c380155-5cf9-4504-adf8-2a03186cf100 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43bb326fb82e4d2c83fb2eaf91860dc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:31.750346 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c380155-5cf9-4504-adf8-2a03186cf100 service cinder] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:31.750916 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c380155-5cf9-4504-adf8-2a03186cf100 service cinder] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:31.751399 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c380155-5cf9-4504-adf8-2a03186cf100 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:31.773730 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c380155-5cf9-4504-adf8-2a03186cf100 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:31.781002 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c380155-5cf9-4504-adf8-2a03186cf100 service cinder] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:31.810811 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 889/1795] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:30:31 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:30:33.270776 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eafddde-555d-4217-8e9e-865edba7f27f None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:33.272303 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-4eafddde-555d-4217-8e9e-865edba7f27f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:33.486649 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eafddde-555d-4217-8e9e-865edba7f27f None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:33.496650 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfabab13-68d5-499e-a1f6-74ba9e201a21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:33.499405 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfabab13-68d5-499e-a1f6-74ba9e201a21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:33.501374 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfabab13-68d5-499e-a1f6-74ba9e201a21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:33.502682 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfabab13-68d5-499e-a1f6-74ba9e201a21 admin admin] PATH_INFO: `/v3/users/f4d23fe9a786498b9cf7e00cb5b983a8` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:33.505980 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfabab13-68d5-499e-a1f6-74ba9e201a21 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f4d23fe9a786498b9cf7e00cb5b983a8)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:33.515006 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfabab13-68d5-499e-a1f6-74ba9e201a21 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:33.679525 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-cfabab13-68d5-499e-a1f6-74ba9e201a21 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f4d23fe9a786498b9cf7e00cb5b983a8'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:33.706378 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-cfabab13-68d5-499e-a1f6-74ba9e201a21 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f4d23fe9a786498b9cf7e00cb5b983a8'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:33.707060 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-cfabab13-68d5-499e-a1f6-74ba9e201a21 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f4d23fe9a786498b9cf7e00cb5b983a8'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:33.724353 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-cfabab13-68d5-499e-a1f6-74ba9e201a21 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f4d23fe9a786498b9cf7e00cb5b983a8'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:33.725560 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-cfabab13-68d5-499e-a1f6-74ba9e201a21 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f4d23fe9a786498b9cf7e00cb5b983a8'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:33.781383 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 907/1796] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Nov 2 07:30:33 2022] DELETE /identity/v3/users/f4d23fe9a786498b9cf7e00cb5b983a8 => generated 0 bytes in 511 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:30:33.785835 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46224bc1-3dc9-45f5-9460-159c9fdc7395 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:33.786413 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46224bc1-3dc9-45f5-9460-159c9fdc7395 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:33.786413 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46224bc1-3dc9-45f5-9460-159c9fdc7395 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:33.807139 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:30:33.864127 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-46224bc1-3dc9-45f5-9460-159c9fdc7395 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:30:33.865416 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-46224bc1-3dc9-45f5-9460-159c9fdc7395 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:34.141373 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 890/1797] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:30:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 356 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:30:34.162165 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfabab13-68d5-499e-a1f6-74ba9e201a21 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:34.163725 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-cfabab13-68d5-499e-a1f6-74ba9e201a21 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:34.275271 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:30:34.316101 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfabab13-68d5-499e-a1f6-74ba9e201a21 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:34.329072 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6241f1b-d58d-4256-a2ec-a21b7a7ab897 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3ec6ecb85f94ef59762877d9ea33ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:34.329697 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6241f1b-d58d-4256-a2ec-a21b7a7ab897 service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:34.329995 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6241f1b-d58d-4256-a2ec-a21b7a7ab897 service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:34.330255 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6241f1b-d58d-4256-a2ec-a21b7a7ab897 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:34.350724 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6241f1b-d58d-4256-a2ec-a21b7a7ab897 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:34.352200 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6241f1b-d58d-4256-a2ec-a21b7a7ab897 service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:34.386769 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 908/1798] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:30:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:30:35.836916 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46224bc1-3dc9-45f5-9460-159c9fdc7395 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:35.838718 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-46224bc1-3dc9-45f5-9460-159c9fdc7395 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:35.857559 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46224bc1-3dc9-45f5-9460-159c9fdc7395 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:35.870831 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b255450c-7db4-41a3-9181-c280a7860a0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:35.871684 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b255450c-7db4-41a3-9181-c280a7860a0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:35.872111 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b255450c-7db4-41a3-9181-c280a7860a0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:35.872715 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b255450c-7db4-41a3-9181-c280a7860a0f admin admin] PATH_INFO: `/v3/projects/d067b8b0214c4e7d8141df0bac1f945e` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:35.875130 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b255450c-7db4-41a3-9181-c280a7860a0f admin admin] RBAC: Authorizing `identity:delete_project(project_id=d067b8b0214c4e7d8141df0bac1f945e)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:35.877042 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b255450c-7db4-41a3-9181-c280a7860a0f admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:35.973309 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-b255450c-7db4-41a3-9181-c280a7860a0f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd067b8b0214c4e7d8141df0bac1f945e', 'request_id': 'req-b255450c-7db4-41a3-9181-c280a7860a0f'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:35.981402 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-b255450c-7db4-41a3-9181-c280a7860a0f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd067b8b0214c4e7d8141df0bac1f945e', 'request_id': 'req-b255450c-7db4-41a3-9181-c280a7860a0f'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:36.007370 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-b255450c-7db4-41a3-9181-c280a7860a0f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd067b8b0214c4e7d8141df0bac1f945e', 'request_id': 'req-b255450c-7db4-41a3-9181-c280a7860a0f'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:36.024522 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-b255450c-7db4-41a3-9181-c280a7860a0f admin admin] The token cache is being invalidate because project d067b8b0214c4e7d8141df0bac1f945e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81501) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:30:36.025244 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-b255450c-7db4-41a3-9181-c280a7860a0f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:36.026327 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-b255450c-7db4-41a3-9181-c280a7860a0f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:36.030406 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 891/1799] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Nov 2 07:30:35 2022] DELETE /identity/v3/projects/d067b8b0214c4e7d8141df0bac1f945e => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:30:36.034799 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6241f1b-d58d-4256-a2ec-a21b7a7ab897 service neutron] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:36.052566 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6241f1b-d58d-4256-a2ec-a21b7a7ab897 service neutron] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:36.190656 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ae3f4e-a35f-4e46-a15a-27576a3f74ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:36.192057 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ae3f4e-a35f-4e46-a15a-27576a3f74ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:36.192661 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ae3f4e-a35f-4e46-a15a-27576a3f74ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:36.193290 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ae3f4e-a35f-4e46-a15a-27576a3f74ef admin admin] PATH_INFO: `/v3/users/f1a726917d034f6db1743bc7ac4be4d5` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:36.196691 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ae3f4e-a35f-4e46-a15a-27576a3f74ef admin admin] RBAC: Authorizing `identity:delete_user(user_id=f1a726917d034f6db1743bc7ac4be4d5)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:36.200087 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ae3f4e-a35f-4e46-a15a-27576a3f74ef admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:36.316278 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-d7ae3f4e-a35f-4e46-a15a-27576a3f74ef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f1a726917d034f6db1743bc7ac4be4d5'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:36.330968 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-d7ae3f4e-a35f-4e46-a15a-27576a3f74ef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f1a726917d034f6db1743bc7ac4be4d5'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:36.331821 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-d7ae3f4e-a35f-4e46-a15a-27576a3f74ef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f1a726917d034f6db1743bc7ac4be4d5'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:36.346328 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-d7ae3f4e-a35f-4e46-a15a-27576a3f74ef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f1a726917d034f6db1743bc7ac4be4d5'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:36.350849 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-d7ae3f4e-a35f-4e46-a15a-27576a3f74ef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f1a726917d034f6db1743bc7ac4be4d5'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:36.398892 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 909/1800] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Nov 2 07:30:36 2022] DELETE /identity/v3/users/f1a726917d034f6db1743bc7ac4be4d5 => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:30:36.944033 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7b3b82-d870-435a-ade8-22c8f15dba59 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:36.944977 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7b3b82-d870-435a-ade8-22c8f15dba59 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:36.945416 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7b3b82-d870-435a-ade8-22c8f15dba59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:37.011004 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-0e7b3b82-d870-435a-ade8-22c8f15dba59 None None] MFA Rules not processed for user `905e2a47c1e34de394b4b380521d364e`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:30:37.012236 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-0e7b3b82-d870-435a-ade8-22c8f15dba59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:37.221517 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ae3f4e-a35f-4e46-a15a-27576a3f74ef admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:37.223203 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-d7ae3f4e-a35f-4e46-a15a-27576a3f74ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:37.224394 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 892/1801] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:30:36 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:30:37.409105 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:30:37.446178 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ae3f4e-a35f-4e46-a15a-27576a3f74ef admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:37.456839 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb80aa2-c596-4aaa-b493-a8937527a507 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e48cba3f804cf08b2477f01ae95179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:37.457717 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb80aa2-c596-4aaa-b493-a8937527a507 service nova] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:37.458710 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb80aa2-c596-4aaa-b493-a8937527a507 service nova] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:37.461934 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb80aa2-c596-4aaa-b493-a8937527a507 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:37.467842 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-3eb80aa2-c596-4aaa-b493-a8937527a507 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:37.583574 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb80aa2-c596-4aaa-b493-a8937527a507 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:37.585278 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb80aa2-c596-4aaa-b493-a8937527a507 service nova] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:37.594536 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7b3b82-d870-435a-ade8-22c8f15dba59 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:37.596264 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-0e7b3b82-d870-435a-ade8-22c8f15dba59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:37.605445 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 910/1802] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:30:37 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:30:37.620563 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:30:37.749980 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7b3b82-d870-435a-ade8-22c8f15dba59 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:37.763409 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5939b24c-0f45-4488-8c9f-c651f802cce5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:37.764380 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5939b24c-0f45-4488-8c9f-c651f802cce5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:37.764764 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5939b24c-0f45-4488-8c9f-c651f802cce5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:37.765412 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5939b24c-0f45-4488-8c9f-c651f802cce5 admin admin] PATH_INFO: `/v3/users/a79c7d7a31074936bc16152715c54dcc` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:37.770550 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5939b24c-0f45-4488-8c9f-c651f802cce5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a79c7d7a31074936bc16152715c54dcc)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:37.773599 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5939b24c-0f45-4488-8c9f-c651f802cce5 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:37.833013 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb80aa2-c596-4aaa-b493-a8937527a507 service nova] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:37.834540 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-3eb80aa2-c596-4aaa-b493-a8937527a507 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:37.882716 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb80aa2-c596-4aaa-b493-a8937527a507 service nova] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:37.929520 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b46024-b5e4-4e3d-917f-b89e5e20ecab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:37.930675 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b46024-b5e4-4e3d-917f-b89e5e20ecab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:37.932473 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b46024-b5e4-4e3d-917f-b89e5e20ecab admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:37.933466 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b46024-b5e4-4e3d-917f-b89e5e20ecab admin admin] PATH_INFO: `/v3/projects/a42f5fc3552c4de683f3d2260e83bcba` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:37.943308 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b46024-b5e4-4e3d-917f-b89e5e20ecab admin admin] RBAC: Authorizing `identity:delete_project(project_id=a42f5fc3552c4de683f3d2260e83bcba)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:37.949571 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b46024-b5e4-4e3d-917f-b89e5e20ecab admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:37.976271 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-5939b24c-0f45-4488-8c9f-c651f802cce5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a79c7d7a31074936bc16152715c54dcc'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:38.012467 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-5939b24c-0f45-4488-8c9f-c651f802cce5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a79c7d7a31074936bc16152715c54dcc'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:38.013790 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-5939b24c-0f45-4488-8c9f-c651f802cce5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a79c7d7a31074936bc16152715c54dcc'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:38.041079 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-5939b24c-0f45-4488-8c9f-c651f802cce5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a79c7d7a31074936bc16152715c54dcc'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:38.135999 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-5939b24c-0f45-4488-8c9f-c651f802cce5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a79c7d7a31074936bc16152715c54dcc'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:38.157835 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 893/1803] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Nov 2 07:30:37 2022] DELETE /identity/v3/users/a79c7d7a31074936bc16152715c54dcc => generated 0 bytes in 564 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:30:38.163199 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcbef80-48ed-4b39-b2f8-82336a747cee None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:38.163659 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcbef80-48ed-4b39-b2f8-82336a747cee None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:38.163860 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcbef80-48ed-4b39-b2f8-82336a747cee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:38.206505 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-f3b46024-b5e4-4e3d-917f-b89e5e20ecab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a42f5fc3552c4de683f3d2260e83bcba', 'request_id': 'req-f3b46024-b5e4-4e3d-917f-b89e5e20ecab'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:38.233270 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-f3b46024-b5e4-4e3d-917f-b89e5e20ecab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a42f5fc3552c4de683f3d2260e83bcba', 'request_id': 'req-f3b46024-b5e4-4e3d-917f-b89e5e20ecab'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:38.242713 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-f3b46024-b5e4-4e3d-917f-b89e5e20ecab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a42f5fc3552c4de683f3d2260e83bcba', 'request_id': 'req-f3b46024-b5e4-4e3d-917f-b89e5e20ecab'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:38.263163 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-0dcbef80-48ed-4b39-b2f8-82336a747cee None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:30:38.272477 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-0dcbef80-48ed-4b39-b2f8-82336a747cee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:38.303709 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-f3b46024-b5e4-4e3d-917f-b89e5e20ecab admin admin] The token cache is being invalidate because project a42f5fc3552c4de683f3d2260e83bcba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:30:38.304539 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-f3b46024-b5e4-4e3d-917f-b89e5e20ecab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:38.309718 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-f3b46024-b5e4-4e3d-917f-b89e5e20ecab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:38.314916 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 911/1804] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Nov 2 07:30:37 2022] DELETE /identity/v3/projects/a42f5fc3552c4de683f3d2260e83bcba => generated 0 bytes in 483 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:30:38.322214 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a5d526-3585-47cb-b8ef-c09e0580bc75 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:38.322810 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a5d526-3585-47cb-b8ef-c09e0580bc75 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:38.323295 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a5d526-3585-47cb-b8ef-c09e0580bc75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:38.362766 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-09a5d526-3585-47cb-b8ef-c09e0580bc75 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:30:38.369779 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-09a5d526-3585-47cb-b8ef-c09e0580bc75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:38.555988 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 894/1805] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:30:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 391 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:30:38.623343 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcbef80-48ed-4b39-b2f8-82336a747cee None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:38.625265 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-0dcbef80-48ed-4b39-b2f8-82336a747cee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:38.665809 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 912/1806] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:30:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 344 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:30:38.674403 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8d0c3e-1642-40d8-8072-2572918b1bc5 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:38.675002 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8d0c3e-1642-40d8-8072-2572918b1bc5 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:38.677831 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8d0c3e-1642-40d8-8072-2572918b1bc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:38.798817 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-0d8d0c3e-1642-40d8-8072-2572918b1bc5 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:30:38.805108 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-0d8d0c3e-1642-40d8-8072-2572918b1bc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:38.807738 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:30:38.873810 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 913/1807] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:30:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:30:38.884542 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8d0c3e-1642-40d8-8072-2572918b1bc5 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:38.909693 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8d0c3e-1642-40d8-8072-2572918b1bc5 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:38.929367 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcbef80-48ed-4b39-b2f8-82336a747cee None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:38.933473 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a0c082-98f7-4afc-9c8b-8caaf3b06a23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:38.936253 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a0c082-98f7-4afc-9c8b-8caaf3b06a23 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:38.937551 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a0c082-98f7-4afc-9c8b-8caaf3b06a23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:38.940141 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a0c082-98f7-4afc-9c8b-8caaf3b06a23 admin admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:38.942078 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a0c082-98f7-4afc-9c8b-8caaf3b06a23 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:30:38.942965 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3962440-3da1-4111-8d5a-31cd03647d7f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3ec6ecb85f94ef59762877d9ea33ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:38.943486 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a0c082-98f7-4afc-9c8b-8caaf3b06a23 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:38.946428 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3962440-3da1-4111-8d5a-31cd03647d7f service neutron] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:38.947906 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3962440-3da1-4111-8d5a-31cd03647d7f service neutron] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:38.949433 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a0c082-98f7-4afc-9c8b-8caaf3b06a23 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:38.949786 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3962440-3da1-4111-8d5a-31cd03647d7f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:38.983259 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3962440-3da1-4111-8d5a-31cd03647d7f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:38.985485 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3962440-3da1-4111-8d5a-31cd03647d7f service neutron] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:38.988011 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:30:38.998502 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 914/1808] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Nov 2 07:30:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:30:39.000832 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343e3485-0dd9-4c71-9591-016b132e6463 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:39.001289 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343e3485-0dd9-4c71-9591-016b132e6463 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:39.001722 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343e3485-0dd9-4c71-9591-016b132e6463 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:39.006405 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 915/1809] 10.222.0.18 () {54 vars in 921 bytes} [Wed Nov 2 07:30:38 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:30:39.008412 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343e3485-0dd9-4c71-9591-016b132e6463 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:39.019671 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 895/1810] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:30:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:30:39.041923 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343e3485-0dd9-4c71-9591-016b132e6463 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:39.066459 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b86540-464a-4815-8928-e7cf29322b7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:39.066459 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b86540-464a-4815-8928-e7cf29322b7e admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:39.066459 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b86540-464a-4815-8928-e7cf29322b7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:39.067980 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b86540-464a-4815-8928-e7cf29322b7e admin admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:39.068653 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b86540-464a-4815-8928-e7cf29322b7e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:39.073792 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b86540-464a-4815-8928-e7cf29322b7e admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:39.116249 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cc7a37-cc6e-4bf1-8b7b-8cc673ac0286 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:39.116525 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cc7a37-cc6e-4bf1-8b7b-8cc673ac0286 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:39.117241 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cc7a37-cc6e-4bf1-8b7b-8cc673ac0286 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:39.165169 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 916/1811] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Nov 2 07:30:39 2022] POST /identity/v3/projects => generated 363 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:30:39.173918 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b86540-464a-4815-8928-e7cf29322b7e admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:39.179817 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-27cc7a37-cc6e-4bf1-8b7b-8cc673ac0286 None None] MFA Rules not processed for user `4f2fc6a156c54d0c9d325901d0f7f6fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:30:39.184390 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-27cc7a37-cc6e-4bf1-8b7b-8cc673ac0286 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:39.501238 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b86540-464a-4815-8928-e7cf29322b7e admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:39.521798 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 896/1812] 10.222.0.18 () {58 vars in 1018 bytes} [Wed Nov 2 07:30:39 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 407 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:30:39.522921 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cc7a37-cc6e-4bf1-8b7b-8cc673ac0286 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:39.525373 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bfed4a-cc69-46c1-97f1-e043e8cae6b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:39.526402 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bfed4a-cc69-46c1-97f1-e043e8cae6b5 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:39.528018 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bfed4a-cc69-46c1-97f1-e043e8cae6b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:39.528691 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bfed4a-cc69-46c1-97f1-e043e8cae6b5 admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:39.529030 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-27cc7a37-cc6e-4bf1-8b7b-8cc673ac0286 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:39.530843 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36bfed4a-cc69-46c1-97f1-e043e8cae6b5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:39.542526 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36bfed4a-cc69-46c1-97f1-e043e8cae6b5 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:39.580613 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cc7a37-cc6e-4bf1-8b7b-8cc673ac0286 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:39.602108 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e813290-e731-4bb2-bf5f-e12ef356daee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:39.607544 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e813290-e731-4bb2-bf5f-e12ef356daee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:39.607544 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e813290-e731-4bb2-bf5f-e12ef356daee admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:39.611933 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e813290-e731-4bb2-bf5f-e12ef356daee admin admin] PATH_INFO: `/v3/users/17a6f8d01e6d433ca467238f07ec15ba` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:39.619590 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 917/1813] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Nov 2 07:30:39 2022] POST /identity/v3/users => generated 282 bytes in 448 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:30:39.621403 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e813290-e731-4bb2-bf5f-e12ef356daee admin admin] RBAC: Authorizing `identity:delete_user(user_id=17a6f8d01e6d433ca467238f07ec15ba)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:39.627761 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bfed4a-cc69-46c1-97f1-e043e8cae6b5 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:39.635414 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e813290-e731-4bb2-bf5f-e12ef356daee admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:39.637173 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-36bfed4a-cc69-46c1-97f1-e043e8cae6b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:39.716434 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bfed4a-cc69-46c1-97f1-e043e8cae6b5 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:39.741781 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba4bb53-2294-407d-8791-7d278a38db7b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2fc6a156c54d0c9d325901d0f7f6fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:39.745454 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba4bb53-2294-407d-8791-7d278a38db7b service placement] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:39.746574 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba4bb53-2294-407d-8791-7d278a38db7b service placement] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:39.750419 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba4bb53-2294-407d-8791-7d278a38db7b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:39.784056 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba4bb53-2294-407d-8791-7d278a38db7b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:39.790079 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba4bb53-2294-407d-8791-7d278a38db7b service placement] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:39.852758 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 918/1814] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:30:39 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:30:39.857472 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba4bb53-2294-407d-8791-7d278a38db7b service placement] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:39.909862 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-0e813290-e731-4bb2-bf5f-e12ef356daee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '17a6f8d01e6d433ca467238f07ec15ba'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:39.913373 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba4bb53-2294-407d-8791-7d278a38db7b service placement] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:39.936002 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-0e813290-e731-4bb2-bf5f-e12ef356daee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '17a6f8d01e6d433ca467238f07ec15ba'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:39.936685 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-0e813290-e731-4bb2-bf5f-e12ef356daee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '17a6f8d01e6d433ca467238f07ec15ba'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:39.945421 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfba1bdd-2f49-44bf-89dc-e1984cf5990e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:39.946861 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfba1bdd-2f49-44bf-89dc-e1984cf5990e admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:39.946861 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfba1bdd-2f49-44bf-89dc-e1984cf5990e admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:39.946861 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfba1bdd-2f49-44bf-89dc-e1984cf5990e admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:39.947371 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfba1bdd-2f49-44bf-89dc-e1984cf5990e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:39.961890 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfba1bdd-2f49-44bf-89dc-e1984cf5990e admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:39.965540 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-0e813290-e731-4bb2-bf5f-e12ef356daee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '17a6f8d01e6d433ca467238f07ec15ba'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:40.013825 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 919/1815] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:30:39 2022] GET /identity/v3/roles => generated 1420 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:30:40.018220 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfba1bdd-2f49-44bf-89dc-e1984cf5990e admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:40.030842 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-bfba1bdd-2f49-44bf-89dc-e1984cf5990e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:40.051545 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-0e813290-e731-4bb2-bf5f-e12ef356daee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '17a6f8d01e6d433ca467238f07ec15ba'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:40.067713 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfba1bdd-2f49-44bf-89dc-e1984cf5990e admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:40.081393 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 897/1816] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Nov 2 07:30:39 2022] DELETE /identity/v3/users/17a6f8d01e6d433ca467238f07ec15ba => generated 0 bytes in 559 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:30:40.088432 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ecd2a8-6223-4a18-8719-4ad519049423 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:40.089072 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ecd2a8-6223-4a18-8719-4ad519049423 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:40.089354 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ecd2a8-6223-4a18-8719-4ad519049423 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:40.089622 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ecd2a8-6223-4a18-8719-4ad519049423 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:40.090276 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ecd2a8-6223-4a18-8719-4ad519049423 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:40.095484 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ecd2a8-6223-4a18-8719-4ad519049423 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:40.137430 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 920/1817] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:30:40 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:30:40.138111 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e813290-e731-4bb2-bf5f-e12ef356daee admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:40.261480 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:30:40.318746 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e813290-e731-4bb2-bf5f-e12ef356daee admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:40.331432 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c117ba-2f1f-4587-95f6-d4a2be6e11b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:40.332648 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c117ba-2f1f-4587-95f6-d4a2be6e11b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:40.332938 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c117ba-2f1f-4587-95f6-d4a2be6e11b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:40.333353 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c117ba-2f1f-4587-95f6-d4a2be6e11b0 admin admin] PATH_INFO: `/v3/projects/a8380aaeff064032bfd507366894f3cb/users/5f1b024711254d34ac54b71f9de3a89b/roles/b494fedd00f24f0b84a529e8672a156a` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:40.410285 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c117ba-2f1f-4587-95f6-d4a2be6e11b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a8380aaeff064032bfd507366894f3cb, user_id=5f1b024711254d34ac54b71f9de3a89b, role_id=b494fedd00f24f0b84a529e8672a156a)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:40.427885 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c117ba-2f1f-4587-95f6-d4a2be6e11b0 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:40.467272 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 898/1818] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Nov 2 07:30:40 2022] PUT /identity/v3/projects/a8380aaeff064032bfd507366894f3cb/users/5f1b024711254d34ac54b71f9de3a89b/roles/b494fedd00f24f0b84a529e8672a156a => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:30:40.473720 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ecd2a8-6223-4a18-8719-4ad519049423 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:40.476285 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-58ecd2a8-6223-4a18-8719-4ad519049423 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:40.504297 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b21fe20-6707-411f-b335-18629bedf937 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:40.504592 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b21fe20-6707-411f-b335-18629bedf937 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:40.504873 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b21fe20-6707-411f-b335-18629bedf937 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:40.523258 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ecd2a8-6223-4a18-8719-4ad519049423 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:40.585944 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-4b21fe20-6707-411f-b335-18629bedf937 None None] MFA Rules not processed for user `5f1b024711254d34ac54b71f9de3a89b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:30:40.587417 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-4b21fe20-6707-411f-b335-18629bedf937 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:40.694528 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563a33ad-d433-4386-98d2-a9cd44c5bd26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:40.695215 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563a33ad-d433-4386-98d2-a9cd44c5bd26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:40.695561 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563a33ad-d433-4386-98d2-a9cd44c5bd26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:40.695898 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563a33ad-d433-4386-98d2-a9cd44c5bd26 admin admin] PATH_INFO: `/v3/projects/f676ec82a80f4425a6fa7562ee47a4cf` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:40.699195 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563a33ad-d433-4386-98d2-a9cd44c5bd26 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f676ec82a80f4425a6fa7562ee47a4cf)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:40.701789 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563a33ad-d433-4386-98d2-a9cd44c5bd26 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:40.777460 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 899/1819] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:30:40 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:30:40.786243 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b21fe20-6707-411f-b335-18629bedf937 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:40.796531 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=94,keystone:INSERT=4,keystone:DELETE=2 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:30:40.814089 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b21fe20-6707-411f-b335-18629bedf937 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:40.820896 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-563a33ad-d433-4386-98d2-a9cd44c5bd26 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f676ec82a80f4425a6fa7562ee47a4cf', 'request_id': 'req-563a33ad-d433-4386-98d2-a9cd44c5bd26'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:40.832111 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-563a33ad-d433-4386-98d2-a9cd44c5bd26 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f676ec82a80f4425a6fa7562ee47a4cf', 'request_id': 'req-563a33ad-d433-4386-98d2-a9cd44c5bd26'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:40.852831 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-563a33ad-d433-4386-98d2-a9cd44c5bd26 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f676ec82a80f4425a6fa7562ee47a4cf', 'request_id': 'req-563a33ad-d433-4386-98d2-a9cd44c5bd26'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:40.882211 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-563a33ad-d433-4386-98d2-a9cd44c5bd26 admin admin] The token cache is being invalidate because project f676ec82a80f4425a6fa7562ee47a4cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:30:40.883613 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-563a33ad-d433-4386-98d2-a9cd44c5bd26 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:40.884488 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-563a33ad-d433-4386-98d2-a9cd44c5bd26 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:40.888055 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 921/1820] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Nov 2 07:30:40 2022] DELETE /identity/v3/projects/f676ec82a80f4425a6fa7562ee47a4cf => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:30:40.894865 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035abd1f-15d1-4685-a1bf-fabd1058dfc7 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:40.897333 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035abd1f-15d1-4685-a1bf-fabd1058dfc7 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:40.900009 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035abd1f-15d1-4685-a1bf-fabd1058dfc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:40.945076 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-035abd1f-15d1-4685-a1bf-fabd1058dfc7 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:30:40.947419 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-035abd1f-15d1-4685-a1bf-fabd1058dfc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:40.972802 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b48c52-54b0-40ad-aa1f-91e0cecc6d81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:40.973599 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b48c52-54b0-40ad-aa1f-91e0cecc6d81 admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:40.975976 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b48c52-54b0-40ad-aa1f-91e0cecc6d81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:40.976434 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b48c52-54b0-40ad-aa1f-91e0cecc6d81 admin admin] PATH_INFO: `/v3/projects` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:40.977267 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b48c52-54b0-40ad-aa1f-91e0cecc6d81 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:40.985522 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b48c52-54b0-40ad-aa1f-91e0cecc6d81 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:41.055104 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 900/1821] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Nov 2 07:30:40 2022] POST /identity/v3/projects => generated 361 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:30:41.057025 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b48c52-54b0-40ad-aa1f-91e0cecc6d81 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:41.061892 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-b9b48c52-54b0-40ad-aa1f-91e0cecc6d81 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:41.107390 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 922/1822] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:30:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:30:41.114945 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620bd4fd-51fc-417c-8d5a-79ca80ae7a79 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:41.116023 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620bd4fd-51fc-417c-8d5a-79ca80ae7a79 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:41.116531 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620bd4fd-51fc-417c-8d5a-79ca80ae7a79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:41.156698 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-620bd4fd-51fc-417c-8d5a-79ca80ae7a79 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:30:41.158100 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-620bd4fd-51fc-417c-8d5a-79ca80ae7a79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:41.161037 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:30:41.218767 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b48c52-54b0-40ad-aa1f-91e0cecc6d81 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:41.250234 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4932bc84-8bdd-4b2d-8a7c-f94c8cc00e03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:41.254801 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4932bc84-8bdd-4b2d-8a7c-f94c8cc00e03 admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:41.255680 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4932bc84-8bdd-4b2d-8a7c-f94c8cc00e03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:41.256106 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4932bc84-8bdd-4b2d-8a7c-f94c8cc00e03 admin admin] PATH_INFO: `/v3/users` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:41.256939 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4932bc84-8bdd-4b2d-8a7c-f94c8cc00e03 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:41.261734 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4932bc84-8bdd-4b2d-8a7c-f94c8cc00e03 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:41.314300 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 901/1823] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Nov 2 07:30:41 2022] POST /identity/v3/users => generated 287 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:30:41.320462 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4932bc84-8bdd-4b2d-8a7c-f94c8cc00e03 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:41.346748 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 923/1824] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:30:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:30:41.346874 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4932bc84-8bdd-4b2d-8a7c-f94c8cc00e03 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:41.348383 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620bd4fd-51fc-417c-8d5a-79ca80ae7a79 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:41.367583 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699efae5-1ea0-4acb-bffa-ff1646fbed78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:41.368637 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699efae5-1ea0-4acb-bffa-ff1646fbed78 admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:41.368917 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699efae5-1ea0-4acb-bffa-ff1646fbed78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:41.369331 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699efae5-1ea0-4acb-bffa-ff1646fbed78 admin admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:41.369841 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699efae5-1ea0-4acb-bffa-ff1646fbed78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:41.370101 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620bd4fd-51fc-417c-8d5a-79ca80ae7a79 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:41.373529 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699efae5-1ea0-4acb-bffa-ff1646fbed78 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:41.379125 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc3278b-c140-4f49-bbe8-5750a0b01835 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:41.380492 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc3278b-c140-4f49-bbe8-5750a0b01835 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:41.380613 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc3278b-c140-4f49-bbe8-5750a0b01835 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:41.380889 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc3278b-c140-4f49-bbe8-5750a0b01835 admin admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:41.381667 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc3278b-c140-4f49-bbe8-5750a0b01835 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:30:41.382330 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc3278b-c140-4f49-bbe8-5750a0b01835 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:41.391449 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc3278b-c140-4f49-bbe8-5750a0b01835 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:41.411796 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 902/1825] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:30:41 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:30:41.416478 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699efae5-1ea0-4acb-bffa-ff1646fbed78 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:41.433757 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 924/1826] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Nov 2 07:30:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:30:41.433928 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699efae5-1ea0-4acb-bffa-ff1646fbed78 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:41.437081 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc3278b-c140-4f49-bbe8-5750a0b01835 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:41.442536 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d33bbba-add2-4fe5-ade2-979052669893 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:41.443382 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d33bbba-add2-4fe5-ade2-979052669893 admin admin] REQUEST_METHOD: `PUT` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:41.444503 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d33bbba-add2-4fe5-ade2-979052669893 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:41.445046 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d33bbba-add2-4fe5-ade2-979052669893 admin admin] PATH_INFO: `/v3/projects/58f8f12d6b96488cb63547e9252af07c/users/d925e68bd0a844ba9d47218573d96a86/roles/c4bf019fc6c44c71a6c984f6d0bab75f` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:41.463963 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc3278b-c140-4f49-bbe8-5750a0b01835 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:41.477393 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b268ee-3d83-4af3-9763-fce91b4e3171 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:41.477679 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d33bbba-add2-4fe5-ade2-979052669893 admin admin] RBAC: Authorizing `identity:create_grant(project_id=58f8f12d6b96488cb63547e9252af07c, user_id=d925e68bd0a844ba9d47218573d96a86, role_id=c4bf019fc6c44c71a6c984f6d0bab75f)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:41.478074 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b268ee-3d83-4af3-9763-fce91b4e3171 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:41.478294 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b268ee-3d83-4af3-9763-fce91b4e3171 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:41.478616 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b268ee-3d83-4af3-9763-fce91b4e3171 admin admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:41.479272 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b268ee-3d83-4af3-9763-fce91b4e3171 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:41.481489 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b268ee-3d83-4af3-9763-fce91b4e3171 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:41.481666 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d33bbba-add2-4fe5-ade2-979052669893 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:41.507501 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 903/1827] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Nov 2 07:30:41 2022] PUT /identity/v3/projects/58f8f12d6b96488cb63547e9252af07c/users/d925e68bd0a844ba9d47218573d96a86/roles/c4bf019fc6c44c71a6c984f6d0bab75f => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:30:41.511601 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d33bbba-add2-4fe5-ade2-979052669893 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:41.547658 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 925/1828] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Nov 2 07:30:41 2022] POST /identity/v3/projects => generated 385 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:30:41.549790 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b268ee-3d83-4af3-9763-fce91b4e3171 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:41.682123 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:30:41.747280 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d33bbba-add2-4fe5-ade2-979052669893 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:41.792794 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b268ee-3d83-4af3-9763-fce91b4e3171 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:41.801391 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7268ea1c-9bbb-44ea-8459-68374db24715 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:41.802954 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7268ea1c-9bbb-44ea-8459-68374db24715 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:41.803343 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7268ea1c-9bbb-44ea-8459-68374db24715 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:41.804767 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7268ea1c-9bbb-44ea-8459-68374db24715 admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:41.805549 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7268ea1c-9bbb-44ea-8459-68374db24715 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:41.808800 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7268ea1c-9bbb-44ea-8459-68374db24715 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:41.843300 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 926/1829] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Nov 2 07:30:41 2022] POST /identity/v3/users => generated 293 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:30:41.844411 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7268ea1c-9bbb-44ea-8459-68374db24715 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:41.849990 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-7268ea1c-9bbb-44ea-8459-68374db24715 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:41.869215 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7268ea1c-9bbb-44ea-8459-68374db24715 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:41.883399 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1fc196-d7f9-40a5-a8c2-aec408691772 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:41.884057 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1fc196-d7f9-40a5-a8c2-aec408691772 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:41.884343 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1fc196-d7f9-40a5-a8c2-aec408691772 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:41.884908 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1fc196-d7f9-40a5-a8c2-aec408691772 admin admin] PATH_INFO: `/v3/projects/e9d315fca9c14f30afdcc8e791a7bfbe` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:41.890268 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1fc196-d7f9-40a5-a8c2-aec408691772 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e9d315fca9c14f30afdcc8e791a7bfbe)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:41.898079 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1fc196-d7f9-40a5-a8c2-aec408691772 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:41.911006 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34b9fcc-8255-4d33-ab28-871f6b94f1b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:41.912001 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34b9fcc-8255-4d33-ab28-871f6b94f1b4 admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:41.912572 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34b9fcc-8255-4d33-ab28-871f6b94f1b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:41.913232 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34b9fcc-8255-4d33-ab28-871f6b94f1b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:41.914124 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34b9fcc-8255-4d33-ab28-871f6b94f1b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:41.916754 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34b9fcc-8255-4d33-ab28-871f6b94f1b4 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:41.932883 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 904/1830] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:30:41 2022] GET /identity/v3/roles => generated 1420 bytes in 422 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:30:41.934098 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34b9fcc-8255-4d33-ab28-871f6b94f1b4 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:41.955139 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34b9fcc-8255-4d33-ab28-871f6b94f1b4 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:41.968113 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92276936-9da4-41f9-ae38-40f88808e070 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:41.969084 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92276936-9da4-41f9-ae38-40f88808e070 admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:41.969610 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92276936-9da4-41f9-ae38-40f88808e070 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:41.970152 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92276936-9da4-41f9-ae38-40f88808e070 admin admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:41.970746 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92276936-9da4-41f9-ae38-40f88808e070 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:41.972429 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92276936-9da4-41f9-ae38-40f88808e070 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:41.990308 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 905/1831] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:30:41 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:30:41.992217 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92276936-9da4-41f9-ae38-40f88808e070 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:42.010481 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92276936-9da4-41f9-ae38-40f88808e070 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:42.025840 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-db1fc196-d7f9-40a5-a8c2-aec408691772 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e9d315fca9c14f30afdcc8e791a7bfbe', 'request_id': 'req-db1fc196-d7f9-40a5-a8c2-aec408691772'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:42.042941 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-db1fc196-d7f9-40a5-a8c2-aec408691772 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e9d315fca9c14f30afdcc8e791a7bfbe', 'request_id': 'req-db1fc196-d7f9-40a5-a8c2-aec408691772'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:42.052731 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-db1fc196-d7f9-40a5-a8c2-aec408691772 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e9d315fca9c14f30afdcc8e791a7bfbe', 'request_id': 'req-db1fc196-d7f9-40a5-a8c2-aec408691772'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:42.085769 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-db1fc196-d7f9-40a5-a8c2-aec408691772 admin admin] The token cache is being invalidate because project e9d315fca9c14f30afdcc8e791a7bfbe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:30:42.086073 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-db1fc196-d7f9-40a5-a8c2-aec408691772 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:42.089063 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:30:42.093950 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-db1fc196-d7f9-40a5-a8c2-aec408691772 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:42.098290 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 927/1832] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Nov 2 07:30:41 2022] DELETE /identity/v3/projects/e9d315fca9c14f30afdcc8e791a7bfbe => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:30:42.099204 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1fc196-d7f9-40a5-a8c2-aec408691772 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:42.100675 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-db1fc196-d7f9-40a5-a8c2-aec408691772 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:42.168326 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5393488e-850c-4ef2-920e-345253044460 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:42.168984 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5393488e-850c-4ef2-920e-345253044460 admin admin] REQUEST_METHOD: `PUT` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:42.169270 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5393488e-850c-4ef2-920e-345253044460 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:42.170114 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5393488e-850c-4ef2-920e-345253044460 admin admin] PATH_INFO: `/v3/projects/58f8f12d6b96488cb63547e9252af07c/users/d925e68bd0a844ba9d47218573d96a86/roles/c4bf019fc6c44c71a6c984f6d0bab75f` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:42.173958 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5393488e-850c-4ef2-920e-345253044460 admin admin] RBAC: Authorizing `identity:create_grant(project_id=58f8f12d6b96488cb63547e9252af07c, user_id=d925e68bd0a844ba9d47218573d96a86, role_id=c4bf019fc6c44c71a6c984f6d0bab75f)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:42.176676 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5393488e-850c-4ef2-920e-345253044460 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:42.201256 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 906/1833] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Nov 2 07:30:41 2022] PUT /identity/v3/projects/58f8f12d6b96488cb63547e9252af07c/users/d925e68bd0a844ba9d47218573d96a86/roles/c4bf019fc6c44c71a6c984f6d0bab75f => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:30:42.215332 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231b76a9-b3e9-4092-bffc-e3926127d0eb None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:42.215627 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231b76a9-b3e9-4092-bffc-e3926127d0eb None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:42.216067 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231b76a9-b3e9-4092-bffc-e3926127d0eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:42.237070 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1fc196-d7f9-40a5-a8c2-aec408691772 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:42.251319 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-231b76a9-b3e9-4092-bffc-e3926127d0eb None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:30:42.252551 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-231b76a9-b3e9-4092-bffc-e3926127d0eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:42.326458 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:30:42.364716 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 907/1834] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:30:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:30:42.365832 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dace5679-5300-4874-ae02-11aea2ebe705 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:42.366271 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dace5679-5300-4874-ae02-11aea2ebe705 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:42.366567 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dace5679-5300-4874-ae02-11aea2ebe705 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:42.372700 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9b3d00-0a30-484e-8c94-1ccc8b672660 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:42.373658 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9b3d00-0a30-484e-8c94-1ccc8b672660 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:42.374178 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9b3d00-0a30-484e-8c94-1ccc8b672660 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:42.374656 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9b3d00-0a30-484e-8c94-1ccc8b672660 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:42.375518 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9b3d00-0a30-484e-8c94-1ccc8b672660 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:42.377475 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9b3d00-0a30-484e-8c94-1ccc8b672660 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:42.399400 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 928/1835] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:30:42 2022] GET /identity/v3/roles => generated 1420 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:30:42.402767 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9beff4-df80-46d4-93a4-7733be7331aa None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:42.403145 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9beff4-df80-46d4-93a4-7733be7331aa None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:42.403655 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9beff4-df80-46d4-93a4-7733be7331aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:42.415767 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-dace5679-5300-4874-ae02-11aea2ebe705 None None] MFA Rules not processed for user `d925e68bd0a844ba9d47218573d96a86`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:30:42.416970 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-dace5679-5300-4874-ae02-11aea2ebe705 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:42.472900 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-df9beff4-df80-46d4-93a4-7733be7331aa None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:30:42.478213 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-df9beff4-df80-46d4-93a4-7733be7331aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:42.512348 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 929/1836] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:30:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:30:42.514176 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9beff4-df80-46d4-93a4-7733be7331aa None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:42.556818 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 908/1837] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:30:42 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:30:42.557445 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dace5679-5300-4874-ae02-11aea2ebe705 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:42.559393 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-dace5679-5300-4874-ae02-11aea2ebe705 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:42.562562 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9beff4-df80-46d4-93a4-7733be7331aa None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:42.579622 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5507a3a2-330b-4a24-aa4f-cfecf83d6892 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:42.581365 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5507a3a2-330b-4a24-aa4f-cfecf83d6892 admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:42.581798 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5507a3a2-330b-4a24-aa4f-cfecf83d6892 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:42.582029 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5507a3a2-330b-4a24-aa4f-cfecf83d6892 admin admin] PATH_INFO: `/v3/projects/ba3bab98d31d49098a8e7242d51be646/users/47965dc7f5964bbfa4efff0822530965/roles/b494fedd00f24f0b84a529e8672a156a` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:42.606641 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dace5679-5300-4874-ae02-11aea2ebe705 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:42.619655 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5507a3a2-330b-4a24-aa4f-cfecf83d6892 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba3bab98d31d49098a8e7242d51be646, user_id=47965dc7f5964bbfa4efff0822530965, role_id=b494fedd00f24f0b84a529e8672a156a)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:42.622285 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2c37f0-f6e5-4153-817a-052e753b8ab4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:42.624378 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5507a3a2-330b-4a24-aa4f-cfecf83d6892 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:42.624700 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2c37f0-f6e5-4153-817a-052e753b8ab4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:42.625184 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2c37f0-f6e5-4153-817a-052e753b8ab4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:42.625657 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2c37f0-f6e5-4153-817a-052e753b8ab4 admin admin] PATH_INFO: `/v3/users/7750009aa9f244cc8ff4ac732eddde5a` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:42.633302 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2c37f0-f6e5-4153-817a-052e753b8ab4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7750009aa9f244cc8ff4ac732eddde5a)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:42.636243 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2c37f0-f6e5-4153-817a-052e753b8ab4 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:42.667853 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 930/1838] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Nov 2 07:30:42 2022] PUT /identity/v3/projects/ba3bab98d31d49098a8e7242d51be646/users/47965dc7f5964bbfa4efff0822530965/roles/b494fedd00f24f0b84a529e8672a156a => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:30:42.672403 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5507a3a2-330b-4a24-aa4f-cfecf83d6892 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:42.792308 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=93,keystone:INSERT=1,keystone:DELETE=6 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:30:42.818486 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-7e2c37f0-f6e5-4153-817a-052e753b8ab4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7750009aa9f244cc8ff4ac732eddde5a'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:42.821273 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5507a3a2-330b-4a24-aa4f-cfecf83d6892 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:42.835212 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-7e2c37f0-f6e5-4153-817a-052e753b8ab4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7750009aa9f244cc8ff4ac732eddde5a'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:42.835896 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-7e2c37f0-f6e5-4153-817a-052e753b8ab4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7750009aa9f244cc8ff4ac732eddde5a'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:42.840305 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3484dd-5191-4cc4-8ccf-6ca57b72757e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:42.845149 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3484dd-5191-4cc4-8ccf-6ca57b72757e admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:42.845551 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3484dd-5191-4cc4-8ccf-6ca57b72757e admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:42.845672 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3484dd-5191-4cc4-8ccf-6ca57b72757e admin admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:42.846843 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3484dd-5191-4cc4-8ccf-6ca57b72757e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:30:42.847151 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3484dd-5191-4cc4-8ccf-6ca57b72757e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:42.848769 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3484dd-5191-4cc4-8ccf-6ca57b72757e admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:42.870212 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-7e2c37f0-f6e5-4153-817a-052e753b8ab4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7750009aa9f244cc8ff4ac732eddde5a'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:42.894243 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 931/1839] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Nov 2 07:30:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:30:42.894967 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3484dd-5191-4cc4-8ccf-6ca57b72757e admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:42.901851 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-7e2c37f0-f6e5-4153-817a-052e753b8ab4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7750009aa9f244cc8ff4ac732eddde5a'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:42.904870 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-7c3484dd-5191-4cc4-8ccf-6ca57b72757e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:42.919790 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 909/1840] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Nov 2 07:30:42 2022] DELETE /identity/v3/users/7750009aa9f244cc8ff4ac732eddde5a => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:30:42.925421 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d90ae6-9d07-4e3c-b2ee-5e4587d0ea7f None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:42.926782 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d90ae6-9d07-4e3c-b2ee-5e4587d0ea7f None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:42.927071 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d90ae6-9d07-4e3c-b2ee-5e4587d0ea7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:42.984282 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-a2d90ae6-9d07-4e3c-b2ee-5e4587d0ea7f None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:30:42.988024 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-a2d90ae6-9d07-4e3c-b2ee-5e4587d0ea7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:43.070303 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3484dd-5191-4cc4-8ccf-6ca57b72757e admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:43.165234 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 910/1841] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:30:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:30:43.166464 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d90ae6-9d07-4e3c-b2ee-5e4587d0ea7f None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:43.171116 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-a2d90ae6-9d07-4e3c-b2ee-5e4587d0ea7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:43.199476 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d90ae6-9d07-4e3c-b2ee-5e4587d0ea7f None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:43.207507 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780398fc-c7dd-41ec-91fb-013fd62eadd2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43bb326fb82e4d2c83fb2eaf91860dc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:43.208814 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780398fc-c7dd-41ec-91fb-013fd62eadd2 service cinder] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:43.209919 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780398fc-c7dd-41ec-91fb-013fd62eadd2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:43.210043 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780398fc-c7dd-41ec-91fb-013fd62eadd2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:43.215120 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913bc5a2-64a6-41ae-a009-ab4e10b3d626 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:43.215677 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913bc5a2-64a6-41ae-a009-ab4e10b3d626 admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:43.215955 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913bc5a2-64a6-41ae-a009-ab4e10b3d626 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:43.216228 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913bc5a2-64a6-41ae-a009-ab4e10b3d626 admin admin] PATH_INFO: `/v3/projects` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:43.218488 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913bc5a2-64a6-41ae-a009-ab4e10b3d626 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:43.221066 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-780398fc-c7dd-41ec-91fb-013fd62eadd2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:43.226179 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913bc5a2-64a6-41ae-a009-ab4e10b3d626 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:43.275639 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:30:43.324138 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 911/1842] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Nov 2 07:30:43 2022] POST /identity/v3/projects => generated 377 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:30:43.330403 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73a0918-5029-45ae-a143-e39d61ac9ae3 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:43.330589 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73a0918-5029-45ae-a143-e39d61ac9ae3 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:43.330928 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73a0918-5029-45ae-a143-e39d61ac9ae3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:43.390090 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-f73a0918-5029-45ae-a143-e39d61ac9ae3 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:30:43.391577 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-f73a0918-5029-45ae-a143-e39d61ac9ae3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:43.426700 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780398fc-c7dd-41ec-91fb-013fd62eadd2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:43.441599 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780398fc-c7dd-41ec-91fb-013fd62eadd2 service cinder] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:43.466500 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 932/1843] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:30:42 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 572 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:30:43.478875 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780398fc-c7dd-41ec-91fb-013fd62eadd2 service cinder] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:43.484491 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-780398fc-c7dd-41ec-91fb-013fd62eadd2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:43.591105 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 912/1844] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:30:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:30:43.592679 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73a0918-5029-45ae-a143-e39d61ac9ae3 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:43.619525 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73a0918-5029-45ae-a143-e39d61ac9ae3 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:43.621641 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780398fc-c7dd-41ec-91fb-013fd62eadd2 service cinder] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:43.650745 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437c165e-1e38-4986-b7ec-3fbd92e14305 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:43.651377 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437c165e-1e38-4986-b7ec-3fbd92e14305 admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:43.655895 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437c165e-1e38-4986-b7ec-3fbd92e14305 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:43.656197 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437c165e-1e38-4986-b7ec-3fbd92e14305 admin admin] PATH_INFO: `/v3/users` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:43.656884 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437c165e-1e38-4986-b7ec-3fbd92e14305 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:43.660001 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437c165e-1e38-4986-b7ec-3fbd92e14305 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:43.713700 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 913/1845] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Nov 2 07:30:43 2022] POST /identity/v3/users => generated 289 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:30:43.713700 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437c165e-1e38-4986-b7ec-3fbd92e14305 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:43.765297 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:30:43.778163 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a734bca2-8bc7-4a65-8084-e33d9186467e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3ec6ecb85f94ef59762877d9ea33ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:43.779068 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a734bca2-8bc7-4a65-8084-e33d9186467e service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:43.780627 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a734bca2-8bc7-4a65-8084-e33d9186467e service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:43.781120 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a734bca2-8bc7-4a65-8084-e33d9186467e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:43.796917 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a734bca2-8bc7-4a65-8084-e33d9186467e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:43.798915 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a734bca2-8bc7-4a65-8084-e33d9186467e service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:43.826690 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 933/1846] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:30:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:30:43.827783 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a734bca2-8bc7-4a65-8084-e33d9186467e service neutron] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:43.840826 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a734bca2-8bc7-4a65-8084-e33d9186467e service neutron] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:43.853252 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3daf2898-3329-446e-9349-933b092e17b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:43.854143 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daf2898-3329-446e-9349-933b092e17b0 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:43.854641 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daf2898-3329-446e-9349-933b092e17b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:43.855025 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daf2898-3329-446e-9349-933b092e17b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:43.855756 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3daf2898-3329-446e-9349-933b092e17b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:43.858548 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3daf2898-3329-446e-9349-933b092e17b0 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:43.862576 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437c165e-1e38-4986-b7ec-3fbd92e14305 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:43.871943 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e0de35-555e-48f7-99f0-ec7009b75549 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3ec6ecb85f94ef59762877d9ea33ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:43.876620 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e0de35-555e-48f7-99f0-ec7009b75549 service neutron] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:43.876925 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e0de35-555e-48f7-99f0-ec7009b75549 service neutron] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:43.877204 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e0de35-555e-48f7-99f0-ec7009b75549 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:43.881773 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 934/1847] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:30:43 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:30:43.886787 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3daf2898-3329-446e-9349-933b092e17b0 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:43.906771 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e0de35-555e-48f7-99f0-ec7009b75549 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:43.908819 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e0de35-555e-48f7-99f0-ec7009b75549 service neutron] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:43.915549 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3daf2898-3329-446e-9349-933b092e17b0 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:43.929931 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004e3aa6-aeaf-4849-bf97-f4291006576e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:43.931974 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004e3aa6-aeaf-4849-bf97-f4291006576e admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:43.932273 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004e3aa6-aeaf-4849-bf97-f4291006576e admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:43.932546 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004e3aa6-aeaf-4849-bf97-f4291006576e admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:43.942792 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004e3aa6-aeaf-4849-bf97-f4291006576e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:43.944846 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 914/1848] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:30:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:30:43.946255 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004e3aa6-aeaf-4849-bf97-f4291006576e admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:44.008200 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 935/1849] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:30:43 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:30:44.009194 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e0de35-555e-48f7-99f0-ec7009b75549 service neutron] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:44.036708 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e0de35-555e-48f7-99f0-ec7009b75549 service neutron] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:44.048811 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fd17a4-a5bc-48be-b651-8bd64e740d1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:44.049762 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fd17a4-a5bc-48be-b651-8bd64e740d1f admin admin] REQUEST_METHOD: `PUT` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:44.050741 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fd17a4-a5bc-48be-b651-8bd64e740d1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:44.053092 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fd17a4-a5bc-48be-b651-8bd64e740d1f admin admin] PATH_INFO: `/v3/projects/f3245fd75d864a34a9644f9d0636dec8/users/094377341ddd4244896a80000070a43b/roles/b494fedd00f24f0b84a529e8672a156a` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:44.096176 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fd17a4-a5bc-48be-b651-8bd64e740d1f admin admin] RBAC: Authorizing `identity:create_grant(project_id=f3245fd75d864a34a9644f9d0636dec8, user_id=094377341ddd4244896a80000070a43b, role_id=b494fedd00f24f0b84a529e8672a156a)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:44.098892 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fd17a4-a5bc-48be-b651-8bd64e740d1f admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:44.138482 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 915/1850] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Nov 2 07:30:44 2022] PUT /identity/v3/projects/f3245fd75d864a34a9644f9d0636dec8/users/094377341ddd4244896a80000070a43b/roles/b494fedd00f24f0b84a529e8672a156a => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:30:44.141305 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bc09b3-3a39-4b22-8054-5db60bdfc15c None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:44.141650 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bc09b3-3a39-4b22-8054-5db60bdfc15c None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:44.141932 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bc09b3-3a39-4b22-8054-5db60bdfc15c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:44.240011 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-72bc09b3-3a39-4b22-8054-5db60bdfc15c None None] MFA Rules not processed for user `094377341ddd4244896a80000070a43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:30:44.240823 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-72bc09b3-3a39-4b22-8054-5db60bdfc15c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:44.319702 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:30:44.330782 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fd17a4-a5bc-48be-b651-8bd64e740d1f admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:44.333660 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-99fd17a4-a5bc-48be-b651-8bd64e740d1f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:44.416407 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 936/1851] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:30:44 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:30:44.421720 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bc09b3-3a39-4b22-8054-5db60bdfc15c None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:44.509971 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fd17a4-a5bc-48be-b651-8bd64e740d1f admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:44.525265 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5836b9e-71cc-4931-bf40-c801d370c1a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:44.526342 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5836b9e-71cc-4931-bf40-c801d370c1a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:44.527952 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5836b9e-71cc-4931-bf40-c801d370c1a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:44.530897 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5836b9e-71cc-4931-bf40-c801d370c1a8 admin admin] PATH_INFO: `/v3/users/5f1b024711254d34ac54b71f9de3a89b` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:44.550732 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5836b9e-71cc-4931-bf40-c801d370c1a8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5f1b024711254d34ac54b71f9de3a89b)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:44.556250 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5836b9e-71cc-4931-bf40-c801d370c1a8 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:44.595770 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bc09b3-3a39-4b22-8054-5db60bdfc15c None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:44.607862 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8be0d70-573e-4180-aab4-1c7ded821d88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:44.608496 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8be0d70-573e-4180-aab4-1c7ded821d88 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:44.608780 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8be0d70-573e-4180-aab4-1c7ded821d88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:44.609155 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8be0d70-573e-4180-aab4-1c7ded821d88 admin admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:44.609801 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8be0d70-573e-4180-aab4-1c7ded821d88 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:44.611969 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8be0d70-573e-4180-aab4-1c7ded821d88 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:44.689044 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 937/1852] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Nov 2 07:30:44 2022] POST /identity/v3/projects => generated 375 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:30:44.690291 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8be0d70-573e-4180-aab4-1c7ded821d88 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:44.760315 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-b5836b9e-71cc-4931-bf40-c801d370c1a8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5f1b024711254d34ac54b71f9de3a89b'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:44.780553 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-b5836b9e-71cc-4931-bf40-c801d370c1a8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5f1b024711254d34ac54b71f9de3a89b'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:44.782142 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-b5836b9e-71cc-4931-bf40-c801d370c1a8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5f1b024711254d34ac54b71f9de3a89b'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:44.797990 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-b5836b9e-71cc-4931-bf40-c801d370c1a8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5f1b024711254d34ac54b71f9de3a89b'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:44.840680 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-b5836b9e-71cc-4931-bf40-c801d370c1a8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5f1b024711254d34ac54b71f9de3a89b'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:44.861710 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 916/1853] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Nov 2 07:30:44 2022] DELETE /identity/v3/users/5f1b024711254d34ac54b71f9de3a89b => generated 0 bytes in 531 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:30:44.867262 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebb2bda-fb8f-458f-bb3b-f49f3d7d226a None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:44.870537 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebb2bda-fb8f-458f-bb3b-f49f3d7d226a None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:44.871544 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebb2bda-fb8f-458f-bb3b-f49f3d7d226a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:44.908224 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8be0d70-573e-4180-aab4-1c7ded821d88 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:44.921580 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-4ebb2bda-fb8f-458f-bb3b-f49f3d7d226a None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:30:44.927623 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-4ebb2bda-fb8f-458f-bb3b-f49f3d7d226a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:44.973251 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:30:45.068550 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b1388c-9e2c-4faf-8764-c6363d3badf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:45.069623 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b1388c-9e2c-4faf-8764-c6363d3badf8 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:45.070396 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b1388c-9e2c-4faf-8764-c6363d3badf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:45.070396 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b1388c-9e2c-4faf-8764-c6363d3badf8 admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:45.071638 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b1388c-9e2c-4faf-8764-c6363d3badf8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:45.077029 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b1388c-9e2c-4faf-8764-c6363d3badf8 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:45.119834 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:30:45.130815 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 938/1854] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Nov 2 07:30:44 2022] POST /identity/v3/users => generated 288 bytes in 438 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:30:45.130815 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b1388c-9e2c-4faf-8764-c6363d3badf8 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:45.135852 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-c3b1388c-9e2c-4faf-8764-c6363d3badf8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:45.153379 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 917/1855] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:30:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 288 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:30:45.157450 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebb2bda-fb8f-458f-bb3b-f49f3d7d226a None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:45.160116 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-4ebb2bda-fb8f-458f-bb3b-f49f3d7d226a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:45.166060 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b1388c-9e2c-4faf-8764-c6363d3badf8 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:45.183168 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f856fbbb-9ec2-4e44-a060-2ba70142b4fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:45.183830 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f856fbbb-9ec2-4e44-a060-2ba70142b4fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:45.184109 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f856fbbb-9ec2-4e44-a060-2ba70142b4fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:45.184380 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f856fbbb-9ec2-4e44-a060-2ba70142b4fb admin admin] PATH_INFO: `/v3/projects/90dfa7f098194bd8bada90ca84f058ca` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:45.184800 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebb2bda-fb8f-458f-bb3b-f49f3d7d226a None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:45.190797 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09d6536-7c43-41ab-8b92-0bd8452f95d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:45.191379 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09d6536-7c43-41ab-8b92-0bd8452f95d1 admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:45.191650 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09d6536-7c43-41ab-8b92-0bd8452f95d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:45.191918 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09d6536-7c43-41ab-8b92-0bd8452f95d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:45.192580 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09d6536-7c43-41ab-8b92-0bd8452f95d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:45.194185 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09d6536-7c43-41ab-8b92-0bd8452f95d1 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:45.196774 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f856fbbb-9ec2-4e44-a060-2ba70142b4fb admin admin] RBAC: Authorizing `identity:delete_project(project_id=90dfa7f098194bd8bada90ca84f058ca)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:45.198467 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f856fbbb-9ec2-4e44-a060-2ba70142b4fb admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:45.210696 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 918/1856] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:30:45 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:30:45.215195 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09d6536-7c43-41ab-8b92-0bd8452f95d1 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:45.234545 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09d6536-7c43-41ab-8b92-0bd8452f95d1 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:45.252551 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b876a16d-8431-4772-b375-aef964888ba3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:45.253226 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b876a16d-8431-4772-b375-aef964888ba3 admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:45.253404 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b876a16d-8431-4772-b375-aef964888ba3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:45.253676 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b876a16d-8431-4772-b375-aef964888ba3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:45.254323 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b876a16d-8431-4772-b375-aef964888ba3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:45.255949 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b876a16d-8431-4772-b375-aef964888ba3 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:45.290959 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 919/1857] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:30:45 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:30:45.291805 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b876a16d-8431-4772-b375-aef964888ba3 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:45.293861 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-b876a16d-8431-4772-b375-aef964888ba3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:45.319901 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-f856fbbb-9ec2-4e44-a060-2ba70142b4fb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '90dfa7f098194bd8bada90ca84f058ca', 'request_id': 'req-f856fbbb-9ec2-4e44-a060-2ba70142b4fb'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:45.334418 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-f856fbbb-9ec2-4e44-a060-2ba70142b4fb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '90dfa7f098194bd8bada90ca84f058ca', 'request_id': 'req-f856fbbb-9ec2-4e44-a060-2ba70142b4fb'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:45.359693 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-f856fbbb-9ec2-4e44-a060-2ba70142b4fb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '90dfa7f098194bd8bada90ca84f058ca', 'request_id': 'req-f856fbbb-9ec2-4e44-a060-2ba70142b4fb'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:45.403607 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-f856fbbb-9ec2-4e44-a060-2ba70142b4fb admin admin] The token cache is being invalidate because project 90dfa7f098194bd8bada90ca84f058ca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:30:45.404280 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-f856fbbb-9ec2-4e44-a060-2ba70142b4fb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:45.405074 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-f856fbbb-9ec2-4e44-a060-2ba70142b4fb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:45.406968 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 939/1858] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Nov 2 07:30:45 2022] DELETE /identity/v3/projects/90dfa7f098194bd8bada90ca84f058ca => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:30:45.410750 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b876a16d-8431-4772-b375-aef964888ba3 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:45.411539 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f856fbbb-9ec2-4e44-a060-2ba70142b4fb admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:45.415805 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-f856fbbb-9ec2-4e44-a060-2ba70142b4fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:45.528151 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7c0d3f-0e0e-4858-a9fc-0909193b295c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3ec6ecb85f94ef59762877d9ea33ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:45.529601 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7c0d3f-0e0e-4858-a9fc-0909193b295c service neutron] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:45.530069 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7c0d3f-0e0e-4858-a9fc-0909193b295c service neutron] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:45.530610 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7c0d3f-0e0e-4858-a9fc-0909193b295c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:45.533459 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-2a7c0d3f-0e0e-4858-a9fc-0909193b295c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:45.619313 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f856fbbb-9ec2-4e44-a060-2ba70142b4fb admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:45.628440 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a72ab2-f12b-48f6-9400-d95f389f99fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:45.630771 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a72ab2-f12b-48f6-9400-d95f389f99fb admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:45.631174 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a72ab2-f12b-48f6-9400-d95f389f99fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:45.631517 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a72ab2-f12b-48f6-9400-d95f389f99fb admin admin] PATH_INFO: `/v3/projects/60d8bc8a38104119870a3716af4bf2e3/users/7d2aeb82e3e5426db63ec411b6ce87cf/roles/b494fedd00f24f0b84a529e8672a156a` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:45.656761 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7c0d3f-0e0e-4858-a9fc-0909193b295c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:45.658794 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7c0d3f-0e0e-4858-a9fc-0909193b295c service neutron] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:45.680947 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 920/1859] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:30:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:30:45.682074 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a72ab2-f12b-48f6-9400-d95f389f99fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=60d8bc8a38104119870a3716af4bf2e3, user_id=7d2aeb82e3e5426db63ec411b6ce87cf, role_id=b494fedd00f24f0b84a529e8672a156a)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:45.690117 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b18748b-acc8-4824-a185-cf66b41f8ee8 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:45.690815 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b18748b-acc8-4824-a185-cf66b41f8ee8 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:45.691441 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a72ab2-f12b-48f6-9400-d95f389f99fb admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:45.692267 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b18748b-acc8-4824-a185-cf66b41f8ee8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:45.737075 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:30:45.753244 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 940/1860] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Nov 2 07:30:45 2022] PUT /identity/v3/projects/60d8bc8a38104119870a3716af4bf2e3/users/7d2aeb82e3e5426db63ec411b6ce87cf/roles/b494fedd00f24f0b84a529e8672a156a => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:30:45.767340 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70355be-bdef-4a32-a77b-11b99cc036dc None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:45.767686 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70355be-bdef-4a32-a77b-11b99cc036dc None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:45.772161 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70355be-bdef-4a32-a77b-11b99cc036dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:45.788058 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-3b18748b-acc8-4824-a185-cf66b41f8ee8 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:30:45.797329 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-3b18748b-acc8-4824-a185-cf66b41f8ee8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:45.865097 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-e70355be-bdef-4a32-a77b-11b99cc036dc None None] MFA Rules not processed for user `7d2aeb82e3e5426db63ec411b6ce87cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:30:45.866921 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-e70355be-bdef-4a32-a77b-11b99cc036dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:46.002695 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 921/1861] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:30:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:30:46.010967 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb3a68c-3078-4988-8ac3-05bdc7c46f27 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:46.011267 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb3a68c-3078-4988-8ac3-05bdc7c46f27 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:46.012826 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb3a68c-3078-4988-8ac3-05bdc7c46f27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:46.030607 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:30:46.076817 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-fdb3a68c-3078-4988-8ac3-05bdc7c46f27 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:30:46.077837 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 941/1862] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:30:45 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 315 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:30:46.080486 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70355be-bdef-4a32-a77b-11b99cc036dc None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:46.080810 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-fdb3a68c-3078-4988-8ac3-05bdc7c46f27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:46.104437 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 922/1863] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:30:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:30:46.110856 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb3a68c-3078-4988-8ac3-05bdc7c46f27 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:46.131936 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb3a68c-3078-4988-8ac3-05bdc7c46f27 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:46.133617 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70355be-bdef-4a32-a77b-11b99cc036dc None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:46.145759 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef1934d-0541-42fd-82a3-4ad6ae5b071c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:46.146382 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef1934d-0541-42fd-82a3-4ad6ae5b071c admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:46.148272 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef1934d-0541-42fd-82a3-4ad6ae5b071c admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:46.148751 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef1934d-0541-42fd-82a3-4ad6ae5b071c admin admin] PATH_INFO: `/v3/domains` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:46.151951 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef1934d-0541-42fd-82a3-4ad6ae5b071c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:30:46.153881 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef1934d-0541-42fd-82a3-4ad6ae5b071c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:46.155763 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef1934d-0541-42fd-82a3-4ad6ae5b071c admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:46.162257 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfbb742-419d-4556-9a13-ba72a085b6f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:46.162659 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfbb742-419d-4556-9a13-ba72a085b6f6 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:46.162938 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfbb742-419d-4556-9a13-ba72a085b6f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:46.163497 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfbb742-419d-4556-9a13-ba72a085b6f6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:46.164166 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfbb742-419d-4556-9a13-ba72a085b6f6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:46.166184 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfbb742-419d-4556-9a13-ba72a085b6f6 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:46.196956 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 923/1864] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Nov 2 07:30:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:30:46.198977 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef1934d-0541-42fd-82a3-4ad6ae5b071c admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:46.213756 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef1934d-0541-42fd-82a3-4ad6ae5b071c admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:46.221797 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e2393f-fbbb-4f8b-a611-e6da06ebbe11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:46.222779 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e2393f-fbbb-4f8b-a611-e6da06ebbe11 admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:46.223711 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e2393f-fbbb-4f8b-a611-e6da06ebbe11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:46.226072 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e2393f-fbbb-4f8b-a611-e6da06ebbe11 admin admin] PATH_INFO: `/v3/projects` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:46.228892 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e2393f-fbbb-4f8b-a611-e6da06ebbe11 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:46.232947 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e2393f-fbbb-4f8b-a611-e6da06ebbe11 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:46.276700 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 942/1865] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Nov 2 07:30:46 2022] POST /identity/v3/projects => generated 375 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:30:46.284060 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfbb742-419d-4556-9a13-ba72a085b6f6 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:46.335477 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 924/1866] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Nov 2 07:30:46 2022] POST /identity/v3/projects => generated 375 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:30:46.339020 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e2393f-fbbb-4f8b-a611-e6da06ebbe11 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:46.469514 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e2393f-fbbb-4f8b-a611-e6da06ebbe11 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:46.475975 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4677fb-7959-49e3-8dc8-931fcf638d46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:46.477395 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4677fb-7959-49e3-8dc8-931fcf638d46 admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:46.477825 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4677fb-7959-49e3-8dc8-931fcf638d46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:46.478841 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4677fb-7959-49e3-8dc8-931fcf638d46 admin admin] PATH_INFO: `/v3/users` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:46.480164 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4677fb-7959-49e3-8dc8-931fcf638d46 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:46.483320 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4677fb-7959-49e3-8dc8-931fcf638d46 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:46.495694 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfbb742-419d-4556-9a13-ba72a085b6f6 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:46.504628 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be62f488-3b76-446f-9e42-28228d7770dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:46.505399 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be62f488-3b76-446f-9e42-28228d7770dc admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:46.505687 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be62f488-3b76-446f-9e42-28228d7770dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:46.506338 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be62f488-3b76-446f-9e42-28228d7770dc admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:46.507124 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be62f488-3b76-446f-9e42-28228d7770dc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:46.511686 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be62f488-3b76-446f-9e42-28228d7770dc admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:46.536338 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 925/1867] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Nov 2 07:30:46 2022] POST /identity/v3/users => generated 288 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 07:30:46.538408 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4677fb-7959-49e3-8dc8-931fcf638d46 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:46.541194 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-dc4677fb-7959-49e3-8dc8-931fcf638d46 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:46.563821 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4677fb-7959-49e3-8dc8-931fcf638d46 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:46.565751 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 943/1868] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Nov 2 07:30:46 2022] POST /identity/v3/users => generated 294 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:30:46.566858 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be62f488-3b76-446f-9e42-28228d7770dc admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:46.571632 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcee24f-46bb-46fb-8452-8ef6437b327b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:46.573139 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcee24f-46bb-46fb-8452-8ef6437b327b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:46.573585 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcee24f-46bb-46fb-8452-8ef6437b327b admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:46.574815 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcee24f-46bb-46fb-8452-8ef6437b327b admin admin] PATH_INFO: `/v3/projects/a8380aaeff064032bfd507366894f3cb` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:46.576479 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfcee24f-46bb-46fb-8452-8ef6437b327b admin admin] RBAC: Authorizing `identity:delete_project(project_id=a8380aaeff064032bfd507366894f3cb)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:46.579293 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfcee24f-46bb-46fb-8452-8ef6437b327b admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:46.593747 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be62f488-3b76-446f-9e42-28228d7770dc admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:46.606240 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5bfc9a-d22c-449a-a807-d20e2c367a7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:46.607748 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5bfc9a-d22c-449a-a807-d20e2c367a7b admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:46.608141 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5bfc9a-d22c-449a-a807-d20e2c367a7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:46.608485 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5bfc9a-d22c-449a-a807-d20e2c367a7b admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:46.609752 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5bfc9a-d22c-449a-a807-d20e2c367a7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:46.611524 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5bfc9a-d22c-449a-a807-d20e2c367a7b admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:46.636128 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 944/1869] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:30:46 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:30:46.637780 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5bfc9a-d22c-449a-a807-d20e2c367a7b admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:46.662856 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5bfc9a-d22c-449a-a807-d20e2c367a7b admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:46.668757 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:30:46.673823 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-bfcee24f-46bb-46fb-8452-8ef6437b327b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a8380aaeff064032bfd507366894f3cb', 'request_id': 'req-bfcee24f-46bb-46fb-8452-8ef6437b327b'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:46.690530 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-bfcee24f-46bb-46fb-8452-8ef6437b327b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a8380aaeff064032bfd507366894f3cb', 'request_id': 'req-bfcee24f-46bb-46fb-8452-8ef6437b327b'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:46.712138 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-bfcee24f-46bb-46fb-8452-8ef6437b327b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a8380aaeff064032bfd507366894f3cb', 'request_id': 'req-bfcee24f-46bb-46fb-8452-8ef6437b327b'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:46.766472 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-bfcee24f-46bb-46fb-8452-8ef6437b327b admin admin] The token cache is being invalidate because project a8380aaeff064032bfd507366894f3cb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81501) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:30:46.767186 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-bfcee24f-46bb-46fb-8452-8ef6437b327b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:46.768743 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-bfcee24f-46bb-46fb-8452-8ef6437b327b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:46.773717 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 926/1870] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Nov 2 07:30:46 2022] DELETE /identity/v3/projects/a8380aaeff064032bfd507366894f3cb => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:30:46.773717 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcee24f-46bb-46fb-8452-8ef6437b327b admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:46.780602 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-bfcee24f-46bb-46fb-8452-8ef6437b327b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:46.879150 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318816e1-6aad-482a-83aa-3589673eea86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:46.879775 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318816e1-6aad-482a-83aa-3589673eea86 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:46.880152 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318816e1-6aad-482a-83aa-3589673eea86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:46.880422 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318816e1-6aad-482a-83aa-3589673eea86 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:46.881213 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318816e1-6aad-482a-83aa-3589673eea86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:46.882842 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318816e1-6aad-482a-83aa-3589673eea86 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:46.911165 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 945/1871] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:30:46 2022] GET /identity/v3/roles => generated 1420 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:30:46.912643 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318816e1-6aad-482a-83aa-3589673eea86 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:46.914813 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-318816e1-6aad-482a-83aa-3589673eea86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:46.926591 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcee24f-46bb-46fb-8452-8ef6437b327b admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:46.941183 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318816e1-6aad-482a-83aa-3589673eea86 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:46.941618 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf256cc-c28c-43ec-a56a-e3a0059f0224 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:46.942444 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf256cc-c28c-43ec-a56a-e3a0059f0224 admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:46.944142 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf256cc-c28c-43ec-a56a-e3a0059f0224 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:46.944620 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf256cc-c28c-43ec-a56a-e3a0059f0224 admin admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:46.945511 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf256cc-c28c-43ec-a56a-e3a0059f0224 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:46.948816 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf256cc-c28c-43ec-a56a-e3a0059f0224 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:46.950236 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6083b90e-d86f-4193-9977-045ac0ebd105 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:46.951482 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6083b90e-d86f-4193-9977-045ac0ebd105 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:46.951763 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6083b90e-d86f-4193-9977-045ac0ebd105 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:46.952178 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6083b90e-d86f-4193-9977-045ac0ebd105 admin admin] PATH_INFO: `/v3/users/d925e68bd0a844ba9d47218573d96a86` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:46.953566 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6083b90e-d86f-4193-9977-045ac0ebd105 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d925e68bd0a844ba9d47218573d96a86)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:46.957032 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6083b90e-d86f-4193-9977-045ac0ebd105 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:46.995114 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 927/1872] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:30:46 2022] GET /identity/v3/roles => generated 1420 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:30:46.996727 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf256cc-c28c-43ec-a56a-e3a0059f0224 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:47.002098 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-6cf256cc-c28c-43ec-a56a-e3a0059f0224 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:47.034445 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf256cc-c28c-43ec-a56a-e3a0059f0224 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:47.055112 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b2a11b-ab73-4dec-88f9-9b007f89ac74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:47.057361 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b2a11b-ab73-4dec-88f9-9b007f89ac74 admin admin] REQUEST_METHOD: `PUT` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:47.058443 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b2a11b-ab73-4dec-88f9-9b007f89ac74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:47.059029 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b2a11b-ab73-4dec-88f9-9b007f89ac74 admin admin] PATH_INFO: `/v3/projects/32ad260aef134e49851252063e7cf960/users/f094282c85e14e2b93c89ef1ab1a98b2/roles/c4bf019fc6c44c71a6c984f6d0bab75f` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:47.087691 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=90,keystone:INSERT=4,keystone:DELETE=6 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:30:47.090318 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-6083b90e-d86f-4193-9977-045ac0ebd105 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd925e68bd0a844ba9d47218573d96a86'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:47.094670 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b2a11b-ab73-4dec-88f9-9b007f89ac74 admin admin] RBAC: Authorizing `identity:create_grant(project_id=32ad260aef134e49851252063e7cf960, user_id=f094282c85e14e2b93c89ef1ab1a98b2, role_id=c4bf019fc6c44c71a6c984f6d0bab75f)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:47.098597 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b2a11b-ab73-4dec-88f9-9b007f89ac74 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:47.112309 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-6083b90e-d86f-4193-9977-045ac0ebd105 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd925e68bd0a844ba9d47218573d96a86'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:47.113026 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-6083b90e-d86f-4193-9977-045ac0ebd105 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd925e68bd0a844ba9d47218573d96a86'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:47.122729 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-6083b90e-d86f-4193-9977-045ac0ebd105 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd925e68bd0a844ba9d47218573d96a86'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:47.123582 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-6083b90e-d86f-4193-9977-045ac0ebd105 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd925e68bd0a844ba9d47218573d96a86'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:47.129501 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 928/1873] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Nov 2 07:30:46 2022] PUT /identity/v3/projects/32ad260aef134e49851252063e7cf960/users/f094282c85e14e2b93c89ef1ab1a98b2/roles/c4bf019fc6c44c71a6c984f6d0bab75f => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:30:47.133172 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b2a11b-ab73-4dec-88f9-9b007f89ac74 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:47.136606 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-f9b2a11b-ab73-4dec-88f9-9b007f89ac74 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:47.165481 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 946/1874] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Nov 2 07:30:46 2022] DELETE /identity/v3/users/d925e68bd0a844ba9d47218573d96a86 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:30:47.165481 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6083b90e-d86f-4193-9977-045ac0ebd105 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:47.178183 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-6083b90e-d86f-4193-9977-045ac0ebd105 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:47.294340 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b2a11b-ab73-4dec-88f9-9b007f89ac74 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:47.381593 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6083b90e-d86f-4193-9977-045ac0ebd105 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:47.396489 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3504c6c9-263a-4ed6-a4b3-d442d7b336c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:47.398197 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3504c6c9-263a-4ed6-a4b3-d442d7b336c0 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:47.399632 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3504c6c9-263a-4ed6-a4b3-d442d7b336c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:47.400124 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3504c6c9-263a-4ed6-a4b3-d442d7b336c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:47.401028 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3504c6c9-263a-4ed6-a4b3-d442d7b336c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:47.404893 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3504c6c9-263a-4ed6-a4b3-d442d7b336c0 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:47.450977 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 947/1875] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:30:47 2022] GET /identity/v3/roles => generated 1420 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:30:47.453384 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3504c6c9-263a-4ed6-a4b3-d442d7b336c0 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:47.474401 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3504c6c9-263a-4ed6-a4b3-d442d7b336c0 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:47.477196 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:DELETE=4,keystone:SELECT=94,keystone:INSERT=2 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:30:47.483263 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d5c4e0-7c87-4c93-a1f0-c1501f6b5eb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:47.484113 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d5c4e0-7c87-4c93-a1f0-c1501f6b5eb4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:47.484694 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d5c4e0-7c87-4c93-a1f0-c1501f6b5eb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:47.485228 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d5c4e0-7c87-4c93-a1f0-c1501f6b5eb4 admin admin] PATH_INFO: `/v3/projects/32ad260aef134e49851252063e7cf960/users/f094282c85e14e2b93c89ef1ab1a98b2/roles/c4bf019fc6c44c71a6c984f6d0bab75f` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:47.490309 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d5c4e0-7c87-4c93-a1f0-c1501f6b5eb4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=32ad260aef134e49851252063e7cf960, user_id=f094282c85e14e2b93c89ef1ab1a98b2, role_id=c4bf019fc6c44c71a6c984f6d0bab75f)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:47.496364 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d5c4e0-7c87-4c93-a1f0-c1501f6b5eb4 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:47.506557 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c0a444-e9b7-4b9a-9fbf-01f2b80b409b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:47.507838 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c0a444-e9b7-4b9a-9fbf-01f2b80b409b admin admin] REQUEST_METHOD: `PUT` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:47.508577 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c0a444-e9b7-4b9a-9fbf-01f2b80b409b admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:47.508864 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c0a444-e9b7-4b9a-9fbf-01f2b80b409b admin admin] PATH_INFO: `/v3/projects/3040390b4c004b9faae74c42e65d4660/users/9978c2003c684acab802c2d2b29b19ff/roles/b494fedd00f24f0b84a529e8672a156a` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:47.523961 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 948/1876] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Nov 2 07:30:47 2022] PUT /identity/v3/projects/32ad260aef134e49851252063e7cf960/users/f094282c85e14e2b93c89ef1ab1a98b2/roles/c4bf019fc6c44c71a6c984f6d0bab75f => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:30:47.528030 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f99d0c-24b8-4646-bbcf-8c6a4a65d830 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:47.529502 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f99d0c-24b8-4646-bbcf-8c6a4a65d830 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:47.532268 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f99d0c-24b8-4646-bbcf-8c6a4a65d830 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:47.570017 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c0a444-e9b7-4b9a-9fbf-01f2b80b409b admin admin] RBAC: Authorizing `identity:create_grant(project_id=3040390b4c004b9faae74c42e65d4660, user_id=9978c2003c684acab802c2d2b29b19ff, role_id=b494fedd00f24f0b84a529e8672a156a)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:47.574126 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c0a444-e9b7-4b9a-9fbf-01f2b80b409b admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:47.578708 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-b8f99d0c-24b8-4646-bbcf-8c6a4a65d830 None None] MFA Rules not processed for user `f094282c85e14e2b93c89ef1ab1a98b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:30:47.580771 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-b8f99d0c-24b8-4646-bbcf-8c6a4a65d830 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:47.614811 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 929/1877] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Nov 2 07:30:47 2022] PUT /identity/v3/projects/3040390b4c004b9faae74c42e65d4660/users/9978c2003c684acab802c2d2b29b19ff/roles/b494fedd00f24f0b84a529e8672a156a => generated 0 bytes in 475 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:30:47.616378 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bd5967-972b-4b78-b297-f7f99b9a7ab2 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:47.616669 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bd5967-972b-4b78-b297-f7f99b9a7ab2 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:47.616943 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bd5967-972b-4b78-b297-f7f99b9a7ab2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:47.662724 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-76bd5967-972b-4b78-b297-f7f99b9a7ab2 None None] MFA Rules not processed for user `9978c2003c684acab802c2d2b29b19ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:30:47.664670 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-76bd5967-972b-4b78-b297-f7f99b9a7ab2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:47.760511 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 949/1878] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:30:47 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:30:47.780893 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f99d0c-24b8-4646-bbcf-8c6a4a65d830 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:47.787898 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-b8f99d0c-24b8-4646-bbcf-8c6a4a65d830 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:47.864382 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:DELETE=4,keystone:SELECT=94,keystone:INSERT=2 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:30:47.879896 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 930/1879] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:30:47 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:30:47.891907 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bd5967-972b-4b78-b297-f7f99b9a7ab2 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:47.896291 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-76bd5967-972b-4b78-b297-f7f99b9a7ab2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:47.948395 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f99d0c-24b8-4646-bbcf-8c6a4a65d830 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:47.995476 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bd5967-972b-4b78-b297-f7f99b9a7ab2 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:48.007632 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de24377-d026-44dc-870d-15a57abf2c84 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43bb326fb82e4d2c83fb2eaf91860dc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:48.008257 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de24377-d026-44dc-870d-15a57abf2c84 service cinder] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:48.008538 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de24377-d026-44dc-870d-15a57abf2c84 service cinder] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:48.008808 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de24377-d026-44dc-870d-15a57abf2c84 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:48.028727 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de24377-d026-44dc-870d-15a57abf2c84 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:48.030947 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de24377-d026-44dc-870d-15a57abf2c84 service cinder] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:48.043887 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 931/1880] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:30:47 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 07:30:48.058680 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e25439-fdf4-474f-8dac-bfe00edee2de service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43bb326fb82e4d2c83fb2eaf91860dc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:48.061616 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e25439-fdf4-474f-8dac-bfe00edee2de service cinder] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:48.062154 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e25439-fdf4-474f-8dac-bfe00edee2de service cinder] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:48.062875 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e25439-fdf4-474f-8dac-bfe00edee2de service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:48.066695 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-e3e25439-fdf4-474f-8dac-bfe00edee2de service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:48.189695 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e25439-fdf4-474f-8dac-bfe00edee2de service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:48.191807 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e25439-fdf4-474f-8dac-bfe00edee2de service cinder] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:48.207650 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 950/1881] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:30:47 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:30:48.375022 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de24377-d026-44dc-870d-15a57abf2c84 service cinder] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:48.386054 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-4de24377-d026-44dc-870d-15a57abf2c84 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:48.683910 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de24377-d026-44dc-870d-15a57abf2c84 service cinder] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:48.764273 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c4949e-be47-40b5-870b-93a998e830d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:48.764938 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c4949e-be47-40b5-870b-93a998e830d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:48.765756 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c4949e-be47-40b5-870b-93a998e830d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:48.766032 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c4949e-be47-40b5-870b-93a998e830d4 admin admin] PATH_INFO: `/v3/projects/58f8f12d6b96488cb63547e9252af07c` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:48.767942 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c4949e-be47-40b5-870b-93a998e830d4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=58f8f12d6b96488cb63547e9252af07c)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:48.777622 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c4949e-be47-40b5-870b-93a998e830d4 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:48.860532 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:30:49.028572 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-33c4949e-be47-40b5-870b-93a998e830d4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '58f8f12d6b96488cb63547e9252af07c', 'request_id': 'req-33c4949e-be47-40b5-870b-93a998e830d4'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:49.047366 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-33c4949e-be47-40b5-870b-93a998e830d4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '58f8f12d6b96488cb63547e9252af07c', 'request_id': 'req-33c4949e-be47-40b5-870b-93a998e830d4'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:49.057350 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-33c4949e-be47-40b5-870b-93a998e830d4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '58f8f12d6b96488cb63547e9252af07c', 'request_id': 'req-33c4949e-be47-40b5-870b-93a998e830d4'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:49.083600 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-33c4949e-be47-40b5-870b-93a998e830d4 admin admin] The token cache is being invalidate because project 58f8f12d6b96488cb63547e9252af07c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81501) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:30:49.083896 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-33c4949e-be47-40b5-870b-93a998e830d4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:49.087418 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-33c4949e-be47-40b5-870b-93a998e830d4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:49.089270 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 932/1882] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Nov 2 07:30:48 2022] DELETE /identity/v3/projects/58f8f12d6b96488cb63547e9252af07c => generated 0 bytes in 715 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:30:49.094752 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a595f6c5-a54c-4d6f-9b27-3e5f863ce3f4 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:49.095293 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a595f6c5-a54c-4d6f-9b27-3e5f863ce3f4 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:49.095293 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a595f6c5-a54c-4d6f-9b27-3e5f863ce3f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:49.143730 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-a595f6c5-a54c-4d6f-9b27-3e5f863ce3f4 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:30:49.144948 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-a595f6c5-a54c-4d6f-9b27-3e5f863ce3f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:49.359499 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 951/1883] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:30:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:30:49.368269 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9161489c-5f25-49bb-a248-f12e956e1abf None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:49.368823 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9161489c-5f25-49bb-a248-f12e956e1abf None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:49.369099 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9161489c-5f25-49bb-a248-f12e956e1abf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:49.408111 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-9161489c-5f25-49bb-a248-f12e956e1abf None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:30:49.409878 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-9161489c-5f25-49bb-a248-f12e956e1abf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:49.437025 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 933/1884] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:30:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:30:49.441523 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a595f6c5-a54c-4d6f-9b27-3e5f863ce3f4 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:49.465135 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a595f6c5-a54c-4d6f-9b27-3e5f863ce3f4 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:49.478996 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7705b7d-d030-460f-b663-2a96429902e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:49.479628 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7705b7d-d030-460f-b663-2a96429902e4 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:49.479911 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7705b7d-d030-460f-b663-2a96429902e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:49.480188 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7705b7d-d030-460f-b663-2a96429902e4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:49.480859 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7705b7d-d030-460f-b663-2a96429902e4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:30:49.482791 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7705b7d-d030-460f-b663-2a96429902e4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:49.488670 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7705b7d-d030-460f-b663-2a96429902e4 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:49.520571 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 952/1885] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Nov 2 07:30:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:30:49.525299 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9161489c-5f25-49bb-a248-f12e956e1abf None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:49.545484 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9161489c-5f25-49bb-a248-f12e956e1abf None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:49.551413 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5ca6fb-305a-4d26-a344-5d5c060ab9bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:49.552218 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5ca6fb-305a-4d26-a344-5d5c060ab9bf admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:49.552825 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5ca6fb-305a-4d26-a344-5d5c060ab9bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:49.553079 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5ca6fb-305a-4d26-a344-5d5c060ab9bf admin admin] PATH_INFO: `/v3/projects` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:49.554148 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5ca6fb-305a-4d26-a344-5d5c060ab9bf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:49.561308 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5ca6fb-305a-4d26-a344-5d5c060ab9bf admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:49.611613 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 934/1886] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Nov 2 07:30:49 2022] POST /identity/v3/projects => generated 361 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:30:49.617094 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7705b7d-d030-460f-b663-2a96429902e4 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:49.635386 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7705b7d-d030-460f-b663-2a96429902e4 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:49.713032 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:30:49.862044 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211d73d6-b22b-46a5-9910-3d35c099e4bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:49.863923 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211d73d6-b22b-46a5-9910-3d35c099e4bd admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:49.864219 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211d73d6-b22b-46a5-9910-3d35c099e4bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:49.864494 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211d73d6-b22b-46a5-9910-3d35c099e4bd admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:49.867367 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211d73d6-b22b-46a5-9910-3d35c099e4bd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:49.877583 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211d73d6-b22b-46a5-9910-3d35c099e4bd admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:49.965968 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 953/1887] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Nov 2 07:30:49 2022] POST /identity/v3/users => generated 281 bytes in 350 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 07:30:49.969762 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5ca6fb-305a-4d26-a344-5d5c060ab9bf admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:50.009017 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5ca6fb-305a-4d26-a344-5d5c060ab9bf admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:50.057215 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cd49e9-0d76-48d1-8c02-7af9028e013a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:50.059529 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cd49e9-0d76-48d1-8c02-7af9028e013a admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:50.061693 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cd49e9-0d76-48d1-8c02-7af9028e013a admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:50.062133 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cd49e9-0d76-48d1-8c02-7af9028e013a admin admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:50.064487 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20cd49e9-0d76-48d1-8c02-7af9028e013a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:50.067456 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20cd49e9-0d76-48d1-8c02-7af9028e013a admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:50.102733 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 935/1888] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:30:49 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:30:50.104465 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211d73d6-b22b-46a5-9910-3d35c099e4bd admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:50.145143 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211d73d6-b22b-46a5-9910-3d35c099e4bd admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:50.162266 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4304f680-2ad9-48bd-9a09-93e4f495b956 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:50.162919 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4304f680-2ad9-48bd-9a09-93e4f495b956 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:50.163372 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4304f680-2ad9-48bd-9a09-93e4f495b956 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:50.164020 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4304f680-2ad9-48bd-9a09-93e4f495b956 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:50.164872 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4304f680-2ad9-48bd-9a09-93e4f495b956 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:50.168508 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4304f680-2ad9-48bd-9a09-93e4f495b956 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:50.208816 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 954/1889] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:30:50 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:30:50.217891 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cd49e9-0d76-48d1-8c02-7af9028e013a admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:50.262509 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cd49e9-0d76-48d1-8c02-7af9028e013a admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:50.288826 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca3d505-0b3b-4155-b17f-e0e8e2647f47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:50.293315 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca3d505-0b3b-4155-b17f-e0e8e2647f47 admin admin] REQUEST_METHOD: `PUT` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:50.295715 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca3d505-0b3b-4155-b17f-e0e8e2647f47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:50.296056 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca3d505-0b3b-4155-b17f-e0e8e2647f47 admin admin] PATH_INFO: `/v3/projects/cb0bfaf7195e47ff9c7d13023b50f651/users/7c94ade19eda4e0bbd8f479c47c7a558/roles/b494fedd00f24f0b84a529e8672a156a` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:50.347962 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca3d505-0b3b-4155-b17f-e0e8e2647f47 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb0bfaf7195e47ff9c7d13023b50f651, user_id=7c94ade19eda4e0bbd8f479c47c7a558, role_id=b494fedd00f24f0b84a529e8672a156a)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:50.350804 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca3d505-0b3b-4155-b17f-e0e8e2647f47 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:50.383281 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 936/1890] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Nov 2 07:30:50 2022] PUT /identity/v3/projects/cb0bfaf7195e47ff9c7d13023b50f651/users/7c94ade19eda4e0bbd8f479c47c7a558/roles/b494fedd00f24f0b84a529e8672a156a => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:30:50.404944 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262082d6-20ef-4004-8b81-8d4d6331b441 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:50.410489 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262082d6-20ef-4004-8b81-8d4d6331b441 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:50.410796 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262082d6-20ef-4004-8b81-8d4d6331b441 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:50.503608 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-262082d6-20ef-4004-8b81-8d4d6331b441 None None] MFA Rules not processed for user `7c94ade19eda4e0bbd8f479c47c7a558`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:30:50.510742 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-262082d6-20ef-4004-8b81-8d4d6331b441 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:50.559769 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca3d505-0b3b-4155-b17f-e0e8e2647f47 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:50.562328 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-dca3d505-0b3b-4155-b17f-e0e8e2647f47 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:50.769400 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca3d505-0b3b-4155-b17f-e0e8e2647f47 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:50.792724 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7290210e-ec78-4687-a230-28940674a21a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43bb326fb82e4d2c83fb2eaf91860dc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:50.798172 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7290210e-ec78-4687-a230-28940674a21a service cinder] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:50.800190 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7290210e-ec78-4687-a230-28940674a21a service cinder] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:50.800190 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7290210e-ec78-4687-a230-28940674a21a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:50.811709 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 955/1891] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:30:50 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 406 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:30:50.811899 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-7290210e-ec78-4687-a230-28940674a21a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:50.814897 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262082d6-20ef-4004-8b81-8d4d6331b441 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:51.123789 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:30:51.140508 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262082d6-20ef-4004-8b81-8d4d6331b441 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:51.187365 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7290210e-ec78-4687-a230-28940674a21a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:51.188155 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae54709-9545-4399-ba60-36bab45f1b7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:51.190635 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae54709-9545-4399-ba60-36bab45f1b7f admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:51.193410 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae54709-9545-4399-ba60-36bab45f1b7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:51.199599 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae54709-9545-4399-ba60-36bab45f1b7f admin admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:51.200437 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae54709-9545-4399-ba60-36bab45f1b7f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:51.204629 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae54709-9545-4399-ba60-36bab45f1b7f admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:51.208548 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7290210e-ec78-4687-a230-28940674a21a service cinder] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:51.250164 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 937/1892] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:30:50 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 691 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 07:30:51.264969 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 956/1893] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Nov 2 07:30:50 2022] POST /identity/v3/projects => generated 365 bytes in 456 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:30:51.268779 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7290210e-ec78-4687-a230-28940674a21a service cinder] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:51.285968 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7290210e-ec78-4687-a230-28940674a21a service cinder] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:51.378001 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc94965-9453-4fac-ae92-d7780d0f1437 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:51.378689 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc94965-9453-4fac-ae92-d7780d0f1437 admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:51.379183 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc94965-9453-4fac-ae92-d7780d0f1437 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:51.379461 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc94965-9453-4fac-ae92-d7780d0f1437 admin admin] PATH_INFO: `/v3/users` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:51.380306 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc94965-9453-4fac-ae92-d7780d0f1437 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:51.382825 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc94965-9453-4fac-ae92-d7780d0f1437 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:51.415015 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 938/1894] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Nov 2 07:30:51 2022] POST /identity/v3/users => generated 289 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:30:51.424594 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae54709-9545-4399-ba60-36bab45f1b7f admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:51.445860 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae54709-9545-4399-ba60-36bab45f1b7f admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:51.455949 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556c2abb-e594-4bec-ba83-20c8afb9be81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:51.457943 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556c2abb-e594-4bec-ba83-20c8afb9be81 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:51.458734 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556c2abb-e594-4bec-ba83-20c8afb9be81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:51.459383 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556c2abb-e594-4bec-ba83-20c8afb9be81 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:51.461033 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556c2abb-e594-4bec-ba83-20c8afb9be81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:51.465059 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556c2abb-e594-4bec-ba83-20c8afb9be81 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:51.480896 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 957/1895] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:30:51 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:30:51.485397 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc94965-9453-4fac-ae92-d7780d0f1437 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:51.505768 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc94965-9453-4fac-ae92-d7780d0f1437 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:51.518526 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405c3aab-daf6-4c78-87a7-f0d10c22bdcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:51.519161 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405c3aab-daf6-4c78-87a7-f0d10c22bdcd admin admin] REQUEST_METHOD: `PUT` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:51.519445 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405c3aab-daf6-4c78-87a7-f0d10c22bdcd admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:51.519719 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405c3aab-daf6-4c78-87a7-f0d10c22bdcd admin admin] PATH_INFO: `/v3/projects/b26c66ba1f354a5a94aa914071ed42c1/users/d06e683b2d72463490f4829e32cbab99/roles/c4bf019fc6c44c71a6c984f6d0bab75f` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:51.546230 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405c3aab-daf6-4c78-87a7-f0d10c22bdcd admin admin] RBAC: Authorizing `identity:create_grant(project_id=b26c66ba1f354a5a94aa914071ed42c1, user_id=d06e683b2d72463490f4829e32cbab99, role_id=c4bf019fc6c44c71a6c984f6d0bab75f)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:51.549560 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405c3aab-daf6-4c78-87a7-f0d10c22bdcd admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:51.570800 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 939/1896] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Nov 2 07:30:51 2022] PUT /identity/v3/projects/b26c66ba1f354a5a94aa914071ed42c1/users/d06e683b2d72463490f4829e32cbab99/roles/c4bf019fc6c44c71a6c984f6d0bab75f => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:30:51.575547 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556c2abb-e594-4bec-ba83-20c8afb9be81 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:51.590319 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556c2abb-e594-4bec-ba83-20c8afb9be81 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:51.598446 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:30:51.687497 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5c4318-51ca-4eb2-a942-009b24cd64a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:51.688876 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5c4318-51ca-4eb2-a942-009b24cd64a5 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:51.689223 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5c4318-51ca-4eb2-a942-009b24cd64a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:51.689572 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5c4318-51ca-4eb2-a942-009b24cd64a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:51.690273 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5c4318-51ca-4eb2-a942-009b24cd64a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:51.693112 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5c4318-51ca-4eb2-a942-009b24cd64a5 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:51.711213 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 958/1897] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:30:51 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:30:51.712405 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405c3aab-daf6-4c78-87a7-f0d10c22bdcd admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:51.723637 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405c3aab-daf6-4c78-87a7-f0d10c22bdcd admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:51.730186 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf04caa6-ecd4-458a-ba85-4ca9db2b3f1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:51.731372 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf04caa6-ecd4-458a-ba85-4ca9db2b3f1c admin admin] REQUEST_METHOD: `PUT` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:51.732701 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf04caa6-ecd4-458a-ba85-4ca9db2b3f1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:51.733430 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf04caa6-ecd4-458a-ba85-4ca9db2b3f1c admin admin] PATH_INFO: `/v3/projects/b26c66ba1f354a5a94aa914071ed42c1/users/d06e683b2d72463490f4829e32cbab99/roles/c4bf019fc6c44c71a6c984f6d0bab75f` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:51.736386 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf04caa6-ecd4-458a-ba85-4ca9db2b3f1c admin admin] RBAC: Authorizing `identity:create_grant(project_id=b26c66ba1f354a5a94aa914071ed42c1, user_id=d06e683b2d72463490f4829e32cbab99, role_id=c4bf019fc6c44c71a6c984f6d0bab75f)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:51.739500 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf04caa6-ecd4-458a-ba85-4ca9db2b3f1c admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:51.753528 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 940/1898] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Nov 2 07:30:51 2022] PUT /identity/v3/projects/b26c66ba1f354a5a94aa914071ed42c1/users/d06e683b2d72463490f4829e32cbab99/roles/c4bf019fc6c44c71a6c984f6d0bab75f => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:30:51.765498 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819cb9ad-2a40-416c-8209-b79768f9f50f None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:51.766525 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819cb9ad-2a40-416c-8209-b79768f9f50f None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:51.767011 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819cb9ad-2a40-416c-8209-b79768f9f50f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:51.803871 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-819cb9ad-2a40-416c-8209-b79768f9f50f None None] MFA Rules not processed for user `d06e683b2d72463490f4829e32cbab99`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:30:51.805514 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-819cb9ad-2a40-416c-8209-b79768f9f50f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:51.898545 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 959/1899] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:30:51 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:30:51.905511 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf04caa6-ecd4-458a-ba85-4ca9db2b3f1c admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:51.908034 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-cf04caa6-ecd4-458a-ba85-4ca9db2b3f1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:51.986141 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf04caa6-ecd4-458a-ba85-4ca9db2b3f1c admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:51.992540 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6f2a19-5352-4a1e-86af-50d57d27f6d7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43bb326fb82e4d2c83fb2eaf91860dc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:51.993746 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6f2a19-5352-4a1e-86af-50d57d27f6d7 service cinder] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:51.994950 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6f2a19-5352-4a1e-86af-50d57d27f6d7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:51.994950 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6f2a19-5352-4a1e-86af-50d57d27f6d7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:52.017777 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6f2a19-5352-4a1e-86af-50d57d27f6d7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:52.019645 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6f2a19-5352-4a1e-86af-50d57d27f6d7 service cinder] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:52.039757 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 941/1900] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:30:51 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:30:52.148771 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819cb9ad-2a40-416c-8209-b79768f9f50f None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:52.161508 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819cb9ad-2a40-416c-8209-b79768f9f50f None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:52.167054 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf2ff07-30dd-4f9c-8f6b-87f198a6dd30 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43bb326fb82e4d2c83fb2eaf91860dc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:52.167884 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf2ff07-30dd-4f9c-8f6b-87f198a6dd30 service cinder] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:52.168307 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf2ff07-30dd-4f9c-8f6b-87f198a6dd30 service cinder] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:52.168763 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf2ff07-30dd-4f9c-8f6b-87f198a6dd30 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:52.244964 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf2ff07-30dd-4f9c-8f6b-87f198a6dd30 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:52.247282 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf2ff07-30dd-4f9c-8f6b-87f198a6dd30 service cinder] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:52.257939 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 960/1901] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:30:52 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:30:53.050082 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6f2a19-5352-4a1e-86af-50d57d27f6d7 service cinder] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:53.059596 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6f2a19-5352-4a1e-86af-50d57d27f6d7 service cinder] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:53.064937 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69447c2-ff80-42f7-a62d-17b6610c6561 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43bb326fb82e4d2c83fb2eaf91860dc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:53.066467 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69447c2-ff80-42f7-a62d-17b6610c6561 service cinder] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:53.066897 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69447c2-ff80-42f7-a62d-17b6610c6561 service cinder] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:53.067859 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69447c2-ff80-42f7-a62d-17b6610c6561 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:53.070224 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-b69447c2-ff80-42f7-a62d-17b6610c6561 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:53.147862 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:30:53.172298 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69447c2-ff80-42f7-a62d-17b6610c6561 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:53.173997 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69447c2-ff80-42f7-a62d-17b6610c6561 service cinder] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:53.186799 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 942/1902] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:30:53 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:30:53.287332 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49cb76d-3ab1-436c-9c5b-beb9e05de68b None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:53.287878 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49cb76d-3ab1-436c-9c5b-beb9e05de68b None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:53.287986 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49cb76d-3ab1-436c-9c5b-beb9e05de68b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:53.324544 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-c49cb76d-3ab1-436c-9c5b-beb9e05de68b None None] MFA Rules not processed for user `47965dc7f5964bbfa4efff0822530965`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:30:53.326334 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-c49cb76d-3ab1-436c-9c5b-beb9e05de68b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:53.350752 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:30:53.401577 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 961/1903] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:30:53 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:30:53.415385 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69447c2-ff80-42f7-a62d-17b6610c6561 service cinder] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:53.417038 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-b69447c2-ff80-42f7-a62d-17b6610c6561 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:53.526403 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69447c2-ff80-42f7-a62d-17b6610c6561 service cinder] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:53.539733 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1849dfa6-6e3e-47f9-bf01-564a2846da2d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e48cba3f804cf08b2477f01ae95179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:53.541990 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1849dfa6-6e3e-47f9-bf01-564a2846da2d service nova] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:53.541990 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1849dfa6-6e3e-47f9-bf01-564a2846da2d service nova] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:53.541990 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1849dfa6-6e3e-47f9-bf01-564a2846da2d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:53.559000 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1849dfa6-6e3e-47f9-bf01-564a2846da2d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:53.560767 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1849dfa6-6e3e-47f9-bf01-564a2846da2d service nova] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:53.598741 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 943/1904] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:30:53 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:30:55.617379 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86707409-6ad8-44ad-b413-e18cb85f8fa6 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:55.617904 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86707409-6ad8-44ad-b413-e18cb85f8fa6 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:55.618022 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86707409-6ad8-44ad-b413-e18cb85f8fa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:55.664310 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-86707409-6ad8-44ad-b413-e18cb85f8fa6 None None] MFA Rules not processed for user `47965dc7f5964bbfa4efff0822530965`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:30:55.665780 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-86707409-6ad8-44ad-b413-e18cb85f8fa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:55.687785 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 962/1905] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:30:55 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:31:00.090675 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1849dfa6-6e3e-47f9-bf01-564a2846da2d service nova] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:31:00.158925 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1849dfa6-6e3e-47f9-bf01-564a2846da2d service nova] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:31:00.163682 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbc813b-c10f-472c-8aa3-276f0f1efe3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:31:00.164349 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbc813b-c10f-472c-8aa3-276f0f1efe3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:00.164697 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbc813b-c10f-472c-8aa3-276f0f1efe3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:00.164972 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbc813b-c10f-472c-8aa3-276f0f1efe3a admin admin] PATH_INFO: `/v3/users/7c94ade19eda4e0bbd8f479c47c7a558` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:00.166317 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbc813b-c10f-472c-8aa3-276f0f1efe3a admin admin] RBAC: Authorizing `identity:delete_user(user_id=7c94ade19eda4e0bbd8f479c47c7a558)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:31:00.168265 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbc813b-c10f-472c-8aa3-276f0f1efe3a admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:31:00.218864 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-fcbc813b-c10f-472c-8aa3-276f0f1efe3a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7c94ade19eda4e0bbd8f479c47c7a558'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:31:00.225978 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-fcbc813b-c10f-472c-8aa3-276f0f1efe3a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7c94ade19eda4e0bbd8f479c47c7a558'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:31:00.226608 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-fcbc813b-c10f-472c-8aa3-276f0f1efe3a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7c94ade19eda4e0bbd8f479c47c7a558'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:31:00.234082 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-fcbc813b-c10f-472c-8aa3-276f0f1efe3a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7c94ade19eda4e0bbd8f479c47c7a558'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:31:00.236055 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:31:00.248954 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-fcbc813b-c10f-472c-8aa3-276f0f1efe3a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7c94ade19eda4e0bbd8f479c47c7a558'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:31:00.254950 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 944/1906] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Nov 2 07:31:00 2022] DELETE /identity/v3/users/7c94ade19eda4e0bbd8f479c47c7a558 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:31:00.259434 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fb8cc7-e06b-43c2-a527-5a739a1a5c82 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:00.259898 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fb8cc7-e06b-43c2-a527-5a739a1a5c82 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:00.260010 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fb8cc7-e06b-43c2-a527-5a739a1a5c82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:00.281741 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-29fb8cc7-e06b-43c2-a527-5a739a1a5c82 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:31:00.283025 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-29fb8cc7-e06b-43c2-a527-5a739a1a5c82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:31:00.345426 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 963/1907] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:31:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:31:00.356258 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbc813b-c10f-472c-8aa3-276f0f1efe3a admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:31:00.358397 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-fcbc813b-c10f-472c-8aa3-276f0f1efe3a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:31:00.417600 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbc813b-c10f-472c-8aa3-276f0f1efe3a admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:31:00.421715 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3b0b99-d1e4-4ae1-bccc-170fa68ec115 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3ec6ecb85f94ef59762877d9ea33ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:31:00.422428 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3b0b99-d1e4-4ae1-bccc-170fa68ec115 service neutron] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:00.422712 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3b0b99-d1e4-4ae1-bccc-170fa68ec115 service neutron] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:00.422986 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3b0b99-d1e4-4ae1-bccc-170fa68ec115 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:00.432898 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3b0b99-d1e4-4ae1-bccc-170fa68ec115 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:31:00.434397 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3b0b99-d1e4-4ae1-bccc-170fa68ec115 service neutron] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:31:00.445882 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 945/1908] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:31:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:31:00.721120 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fb8cc7-e06b-43c2-a527-5a739a1a5c82 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:31:00.722809 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-29fb8cc7-e06b-43c2-a527-5a739a1a5c82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:31:00.740183 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fb8cc7-e06b-43c2-a527-5a739a1a5c82 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:31:00.746983 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc30723-cb49-49fb-8e62-4819d19b6b2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:31:00.748933 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc30723-cb49-49fb-8e62-4819d19b6b2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:00.749497 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc30723-cb49-49fb-8e62-4819d19b6b2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:00.749945 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc30723-cb49-49fb-8e62-4819d19b6b2d admin admin] PATH_INFO: `/v3/users/094377341ddd4244896a80000070a43b` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:00.751708 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc30723-cb49-49fb-8e62-4819d19b6b2d admin admin] RBAC: Authorizing `identity:delete_user(user_id=094377341ddd4244896a80000070a43b)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:31:00.756953 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc30723-cb49-49fb-8e62-4819d19b6b2d admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:31:00.821049 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-dbc30723-cb49-49fb-8e62-4819d19b6b2d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '094377341ddd4244896a80000070a43b'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:31:00.830822 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-dbc30723-cb49-49fb-8e62-4819d19b6b2d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '094377341ddd4244896a80000070a43b'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:31:00.832355 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-dbc30723-cb49-49fb-8e62-4819d19b6b2d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '094377341ddd4244896a80000070a43b'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:31:00.844021 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-dbc30723-cb49-49fb-8e62-4819d19b6b2d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '094377341ddd4244896a80000070a43b'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:31:00.845481 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-dbc30723-cb49-49fb-8e62-4819d19b6b2d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '094377341ddd4244896a80000070a43b'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:31:00.858865 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3b0b99-d1e4-4ae1-bccc-170fa68ec115 service neutron] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:31:00.860656 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-5d3b0b99-d1e4-4ae1-bccc-170fa68ec115 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:31:00.889220 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3b0b99-d1e4-4ae1-bccc-170fa68ec115 service neutron] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:31:00.897736 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513d886b-a061-418a-9600-859471435aed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:31:00.898008 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513d886b-a061-418a-9600-859471435aed admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:00.898459 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513d886b-a061-418a-9600-859471435aed admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:00.898708 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513d886b-a061-418a-9600-859471435aed admin admin] PATH_INFO: `/v3/projects/cb0bfaf7195e47ff9c7d13023b50f651` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:00.900967 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513d886b-a061-418a-9600-859471435aed admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb0bfaf7195e47ff9c7d13023b50f651)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:31:00.903422 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513d886b-a061-418a-9600-859471435aed admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:31:00.917718 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 964/1909] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Nov 2 07:31:00 2022] DELETE /identity/v3/users/094377341ddd4244896a80000070a43b => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:31:00.923088 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39732a7f-f0e7-4f50-9ab5-77660c992158 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:00.923395 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39732a7f-f0e7-4f50-9ab5-77660c992158 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:00.923672 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39732a7f-f0e7-4f50-9ab5-77660c992158 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:00.953270 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-39732a7f-f0e7-4f50-9ab5-77660c992158 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:31:00.954588 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-39732a7f-f0e7-4f50-9ab5-77660c992158 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:31:00.980823 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:31:00.983044 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-513d886b-a061-418a-9600-859471435aed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb0bfaf7195e47ff9c7d13023b50f651', 'request_id': 'req-513d886b-a061-418a-9600-859471435aed'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:31:00.991373 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-513d886b-a061-418a-9600-859471435aed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb0bfaf7195e47ff9c7d13023b50f651', 'request_id': 'req-513d886b-a061-418a-9600-859471435aed'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:31:01.007001 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-513d886b-a061-418a-9600-859471435aed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb0bfaf7195e47ff9c7d13023b50f651', 'request_id': 'req-513d886b-a061-418a-9600-859471435aed'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:31:01.022177 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-513d886b-a061-418a-9600-859471435aed admin admin] The token cache is being invalidate because project cb0bfaf7195e47ff9c7d13023b50f651 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81501) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:31:01.024133 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-513d886b-a061-418a-9600-859471435aed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:31:01.024133 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-513d886b-a061-418a-9600-859471435aed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:31:01.037293 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 946/1910] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Nov 2 07:31:00 2022] DELETE /identity/v3/projects/cb0bfaf7195e47ff9c7d13023b50f651 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:31:01.043537 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513d886b-a061-418a-9600-859471435aed admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:31:01.045284 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-513d886b-a061-418a-9600-859471435aed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:31:01.050402 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 965/1911] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:31:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:31:01.057978 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39732a7f-f0e7-4f50-9ab5-77660c992158 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:31:01.059632 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-39732a7f-f0e7-4f50-9ab5-77660c992158 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:31:01.138061 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513d886b-a061-418a-9600-859471435aed admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:31:01.145953 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dc2263-c477-45b6-a244-9a62a3920e19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:31:01.146844 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dc2263-c477-45b6-a244-9a62a3920e19 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:01.147300 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dc2263-c477-45b6-a244-9a62a3920e19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:01.147717 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dc2263-c477-45b6-a244-9a62a3920e19 admin admin] PATH_INFO: `/v3/users/d06e683b2d72463490f4829e32cbab99` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:01.149242 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79dc2263-c477-45b6-a244-9a62a3920e19 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d06e683b2d72463490f4829e32cbab99)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:31:01.151275 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79dc2263-c477-45b6-a244-9a62a3920e19 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:31:01.151491 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39732a7f-f0e7-4f50-9ab5-77660c992158 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:31:01.163688 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:31:01.232107 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108269ac-97bd-49d5-981d-832bcdc2ff05 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3ec6ecb85f94ef59762877d9ea33ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:31:01.233243 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108269ac-97bd-49d5-981d-832bcdc2ff05 service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:01.233982 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108269ac-97bd-49d5-981d-832bcdc2ff05 service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:01.234501 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108269ac-97bd-49d5-981d-832bcdc2ff05 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:01.245115 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-79dc2263-c477-45b6-a244-9a62a3920e19 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd06e683b2d72463490f4829e32cbab99'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:31:01.251081 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108269ac-97bd-49d5-981d-832bcdc2ff05 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:31:01.252646 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108269ac-97bd-49d5-981d-832bcdc2ff05 service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:31:01.253614 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-79dc2263-c477-45b6-a244-9a62a3920e19 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd06e683b2d72463490f4829e32cbab99'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:31:01.254776 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-79dc2263-c477-45b6-a244-9a62a3920e19 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd06e683b2d72463490f4829e32cbab99'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:31:01.263033 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-79dc2263-c477-45b6-a244-9a62a3920e19 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd06e683b2d72463490f4829e32cbab99'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:31:01.264256 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 966/1912] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:31:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:31:01.277543 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-79dc2263-c477-45b6-a244-9a62a3920e19 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd06e683b2d72463490f4829e32cbab99'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:31:01.285350 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 947/1913] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Nov 2 07:31:01 2022] DELETE /identity/v3/users/d06e683b2d72463490f4829e32cbab99 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:31:01.929312 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108269ac-97bd-49d5-981d-832bcdc2ff05 service neutron] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:31:01.950635 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108269ac-97bd-49d5-981d-832bcdc2ff05 service neutron] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:31:01.952661 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dc2263-c477-45b6-a244-9a62a3920e19 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:31:01.954298 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-79dc2263-c477-45b6-a244-9a62a3920e19 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:31:02.045047 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1036b9f9-de2b-4058-a62c-8f32d318c514 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:31:02.047221 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1036b9f9-de2b-4058-a62c-8f32d318c514 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:02.048221 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1036b9f9-de2b-4058-a62c-8f32d318c514 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:02.048754 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1036b9f9-de2b-4058-a62c-8f32d318c514 admin admin] PATH_INFO: `/v3/projects/b26c66ba1f354a5a94aa914071ed42c1` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:02.051247 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1036b9f9-de2b-4058-a62c-8f32d318c514 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b26c66ba1f354a5a94aa914071ed42c1)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:31:02.055494 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1036b9f9-de2b-4058-a62c-8f32d318c514 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:31:02.100203 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dc2263-c477-45b6-a244-9a62a3920e19 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:31:02.113968 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:31:02.114136 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b179f22-9a55-4bca-a46e-3037bb46438e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:31:02.115294 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b179f22-9a55-4bca-a46e-3037bb46438e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:02.121001 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b179f22-9a55-4bca-a46e-3037bb46438e admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:02.121502 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b179f22-9a55-4bca-a46e-3037bb46438e admin admin] PATH_INFO: `/v3/projects/f3245fd75d864a34a9644f9d0636dec8` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:02.126891 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b179f22-9a55-4bca-a46e-3037bb46438e admin admin] RBAC: Authorizing `identity:delete_project(project_id=f3245fd75d864a34a9644f9d0636dec8)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:31:02.131783 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b179f22-9a55-4bca-a46e-3037bb46438e admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:31:02.176560 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-1036b9f9-de2b-4058-a62c-8f32d318c514 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b26c66ba1f354a5a94aa914071ed42c1', 'request_id': 'req-1036b9f9-de2b-4058-a62c-8f32d318c514'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:31:02.188099 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-1036b9f9-de2b-4058-a62c-8f32d318c514 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b26c66ba1f354a5a94aa914071ed42c1', 'request_id': 'req-1036b9f9-de2b-4058-a62c-8f32d318c514'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:31:02.202665 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-1036b9f9-de2b-4058-a62c-8f32d318c514 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b26c66ba1f354a5a94aa914071ed42c1', 'request_id': 'req-1036b9f9-de2b-4058-a62c-8f32d318c514'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:31:02.243670 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-1036b9f9-de2b-4058-a62c-8f32d318c514 admin admin] The token cache is being invalidate because project b26c66ba1f354a5a94aa914071ed42c1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:31:02.244226 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-1036b9f9-de2b-4058-a62c-8f32d318c514 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:31:02.245089 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-1036b9f9-de2b-4058-a62c-8f32d318c514 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:31:02.247007 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 967/1914] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Nov 2 07:31:01 2022] DELETE /identity/v3/projects/b26c66ba1f354a5a94aa914071ed42c1 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:31:02.253343 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23350f59-5c67-4af3-adc4-b8d622551339 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:02.253628 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23350f59-5c67-4af3-adc4-b8d622551339 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:02.253902 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23350f59-5c67-4af3-adc4-b8d622551339 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:02.255771 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-1b179f22-9a55-4bca-a46e-3037bb46438e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f3245fd75d864a34a9644f9d0636dec8', 'request_id': 'req-1b179f22-9a55-4bca-a46e-3037bb46438e'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:31:02.262814 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-1b179f22-9a55-4bca-a46e-3037bb46438e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f3245fd75d864a34a9644f9d0636dec8', 'request_id': 'req-1b179f22-9a55-4bca-a46e-3037bb46438e'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:31:02.281570 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-1b179f22-9a55-4bca-a46e-3037bb46438e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f3245fd75d864a34a9644f9d0636dec8', 'request_id': 'req-1b179f22-9a55-4bca-a46e-3037bb46438e'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:31:02.283461 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-23350f59-5c67-4af3-adc4-b8d622551339 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:31:02.285847 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-23350f59-5c67-4af3-adc4-b8d622551339 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:31:02.309198 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-1b179f22-9a55-4bca-a46e-3037bb46438e admin admin] The token cache is being invalidate because project f3245fd75d864a34a9644f9d0636dec8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81501) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:31:02.309498 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-1b179f22-9a55-4bca-a46e-3037bb46438e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:31:02.310949 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-1b179f22-9a55-4bca-a46e-3037bb46438e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:31:02.313526 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 948/1915] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Nov 2 07:31:01 2022] DELETE /identity/v3/projects/f3245fd75d864a34a9644f9d0636dec8 => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:31:02.317161 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b179f22-9a55-4bca-a46e-3037bb46438e admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:31:02.319017 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-1b179f22-9a55-4bca-a46e-3037bb46438e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:31:02.362865 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:31:02.432349 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 968/1916] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:31:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:31:02.440014 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500ec0eb-efee-469e-aeb3-c4bead396edf None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:02.440440 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500ec0eb-efee-469e-aeb3-c4bead396edf None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:02.440851 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500ec0eb-efee-469e-aeb3-c4bead396edf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:02.446437 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b179f22-9a55-4bca-a46e-3037bb46438e admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:31:02.452645 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef9d705-f7f7-453c-96a9-89a6c735d480 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:31:02.453423 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef9d705-f7f7-453c-96a9-89a6c735d480 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:02.453853 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef9d705-f7f7-453c-96a9-89a6c735d480 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:02.453988 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef9d705-f7f7-453c-96a9-89a6c735d480 admin admin] PATH_INFO: `/v3/users/7d2aeb82e3e5426db63ec411b6ce87cf` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:02.455303 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef9d705-f7f7-453c-96a9-89a6c735d480 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7d2aeb82e3e5426db63ec411b6ce87cf)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:31:02.457245 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef9d705-f7f7-453c-96a9-89a6c735d480 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:31:02.474626 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-500ec0eb-efee-469e-aeb3-c4bead396edf None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:31:02.476343 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-500ec0eb-efee-469e-aeb3-c4bead396edf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:31:02.488062 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 969/1917] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:31:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:31:02.491929 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500ec0eb-efee-469e-aeb3-c4bead396edf None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:31:02.507464 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500ec0eb-efee-469e-aeb3-c4bead396edf None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:31:02.517129 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe994ba-996e-49d5-bf7b-feb9c61b9ee2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:31:02.520816 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe994ba-996e-49d5-bf7b-feb9c61b9ee2 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:02.522273 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe994ba-996e-49d5-bf7b-feb9c61b9ee2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:02.522962 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe994ba-996e-49d5-bf7b-feb9c61b9ee2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:02.525088 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe994ba-996e-49d5-bf7b-feb9c61b9ee2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:31:02.525612 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe994ba-996e-49d5-bf7b-feb9c61b9ee2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:31:02.528703 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe994ba-996e-49d5-bf7b-feb9c61b9ee2 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:31:02.540176 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-2ef9d705-f7f7-453c-96a9-89a6c735d480 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7d2aeb82e3e5426db63ec411b6ce87cf'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:31:02.551487 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 970/1918] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Nov 2 07:31:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:31:02.555839 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-2ef9d705-f7f7-453c-96a9-89a6c735d480 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7d2aeb82e3e5426db63ec411b6ce87cf'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:31:02.555974 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe994ba-996e-49d5-bf7b-feb9c61b9ee2 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:31:02.557533 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-2ef9d705-f7f7-453c-96a9-89a6c735d480 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7d2aeb82e3e5426db63ec411b6ce87cf'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:31:02.572008 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe994ba-996e-49d5-bf7b-feb9c61b9ee2 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:31:02.574177 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-2ef9d705-f7f7-453c-96a9-89a6c735d480 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7d2aeb82e3e5426db63ec411b6ce87cf'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:31:02.581173 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25af3904-caa4-4d22-8956-3605dba19e69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:31:02.582411 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25af3904-caa4-4d22-8956-3605dba19e69 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:02.582837 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25af3904-caa4-4d22-8956-3605dba19e69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:02.583451 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25af3904-caa4-4d22-8956-3605dba19e69 admin admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:02.584633 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25af3904-caa4-4d22-8956-3605dba19e69 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:31:02.588328 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25af3904-caa4-4d22-8956-3605dba19e69 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:31:02.600566 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-2ef9d705-f7f7-453c-96a9-89a6c735d480 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7d2aeb82e3e5426db63ec411b6ce87cf'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:31:02.617756 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 949/1919] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Nov 2 07:31:02 2022] DELETE /identity/v3/users/7d2aeb82e3e5426db63ec411b6ce87cf => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:31:02.639939 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 971/1920] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Nov 2 07:31:02 2022] POST /identity/v3/projects => generated 387 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:31:02.644085 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef9d705-f7f7-453c-96a9-89a6c735d480 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:31:02.645575 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-2ef9d705-f7f7-453c-96a9-89a6c735d480 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:31:02.741778 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef9d705-f7f7-453c-96a9-89a6c735d480 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:31:02.804040 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:31:02.831323 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81f947c-9f26-42f2-9b7f-e94ebef659de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:31:02.832127 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81f947c-9f26-42f2-9b7f-e94ebef659de admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:02.832413 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81f947c-9f26-42f2-9b7f-e94ebef659de admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:02.832684 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81f947c-9f26-42f2-9b7f-e94ebef659de admin admin] PATH_INFO: `/v3/users` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:02.833357 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81f947c-9f26-42f2-9b7f-e94ebef659de admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:31:02.835440 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81f947c-9f26-42f2-9b7f-e94ebef659de admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:31:02.870918 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 950/1921] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Nov 2 07:31:02 2022] POST /identity/v3/users => generated 294 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:31:02.871418 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25af3904-caa4-4d22-8956-3605dba19e69 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:31:02.891851 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25af3904-caa4-4d22-8956-3605dba19e69 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:31:02.897093 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f39c80d-2f40-4e72-aa77-6688cf6f78ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:31:02.897707 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f39c80d-2f40-4e72-aa77-6688cf6f78ce admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:02.898032 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f39c80d-2f40-4e72-aa77-6688cf6f78ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:02.898307 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f39c80d-2f40-4e72-aa77-6688cf6f78ce admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:02.899296 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f39c80d-2f40-4e72-aa77-6688cf6f78ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:31:02.900770 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f39c80d-2f40-4e72-aa77-6688cf6f78ce admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:31:02.914398 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 972/1922] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:31:02 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 07:31:02.916018 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81f947c-9f26-42f2-9b7f-e94ebef659de admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:31:02.927097 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81f947c-9f26-42f2-9b7f-e94ebef659de admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:31:02.931733 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368248eb-27b4-4bd7-a849-4bd7b8f2d75f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:31:02.932348 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368248eb-27b4-4bd7-a849-4bd7b8f2d75f admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:02.932626 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368248eb-27b4-4bd7-a849-4bd7b8f2d75f admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:02.932893 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368248eb-27b4-4bd7-a849-4bd7b8f2d75f admin admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:02.933530 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368248eb-27b4-4bd7-a849-4bd7b8f2d75f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:31:02.935894 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368248eb-27b4-4bd7-a849-4bd7b8f2d75f admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:31:02.949629 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 951/1923] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:31:02 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:31:02.952185 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f39c80d-2f40-4e72-aa77-6688cf6f78ce admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:31:02.963115 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f39c80d-2f40-4e72-aa77-6688cf6f78ce admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:31:02.967993 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25f98d7-3e8c-4885-bb10-0bc39800aabb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:31:02.968665 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25f98d7-3e8c-4885-bb10-0bc39800aabb admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:02.968943 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25f98d7-3e8c-4885-bb10-0bc39800aabb admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:02.969219 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25f98d7-3e8c-4885-bb10-0bc39800aabb admin admin] PATH_INFO: `/v3/projects/3b6b21916b764c9fb3594c366e097f2e/users/25c53e3c82724ab4b997635af0ad79e4/roles/b494fedd00f24f0b84a529e8672a156a` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:02.986437 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25f98d7-3e8c-4885-bb10-0bc39800aabb admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b6b21916b764c9fb3594c366e097f2e, user_id=25c53e3c82724ab4b997635af0ad79e4, role_id=b494fedd00f24f0b84a529e8672a156a)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:31:02.989180 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25f98d7-3e8c-4885-bb10-0bc39800aabb admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:31:03.004251 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 973/1924] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Nov 2 07:31:02 2022] PUT /identity/v3/projects/3b6b21916b764c9fb3594c366e097f2e/users/25c53e3c82724ab4b997635af0ad79e4/roles/b494fedd00f24f0b84a529e8672a156a => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:31:03.015874 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e159bce8-2ef7-4575-93fd-9d9b26e76e6b None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:03.016157 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e159bce8-2ef7-4575-93fd-9d9b26e76e6b None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:03.016423 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e159bce8-2ef7-4575-93fd-9d9b26e76e6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:03.047992 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-e159bce8-2ef7-4575-93fd-9d9b26e76e6b None None] MFA Rules not processed for user `25c53e3c82724ab4b997635af0ad79e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:31:03.049210 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-e159bce8-2ef7-4575-93fd-9d9b26e76e6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:31:03.116124 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 952/1925] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:31:03 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:31:03.121070 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25f98d7-3e8c-4885-bb10-0bc39800aabb admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:31:03.163108 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:DELETE=6,keystone:SELECT=91,keystone:INSERT=3 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:31:03.186624 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25f98d7-3e8c-4885-bb10-0bc39800aabb admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:31:03.191244 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3154c21a-9d01-4560-8363-c9552bf6fcfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:31:03.191850 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3154c21a-9d01-4560-8363-c9552bf6fcfc admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:03.192129 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3154c21a-9d01-4560-8363-c9552bf6fcfc admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:03.192396 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3154c21a-9d01-4560-8363-c9552bf6fcfc admin admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:03.193032 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3154c21a-9d01-4560-8363-c9552bf6fcfc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:31:03.194985 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3154c21a-9d01-4560-8363-c9552bf6fcfc admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:31:03.225414 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 974/1926] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Nov 2 07:31:03 2022] POST /identity/v3/projects => generated 387 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:31:03.229549 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e159bce8-2ef7-4575-93fd-9d9b26e76e6b None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:31:03.243158 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e159bce8-2ef7-4575-93fd-9d9b26e76e6b None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:31:03.320360 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb3feac-55f5-4222-a5e0-2eadd02131c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:31:03.320974 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb3feac-55f5-4222-a5e0-2eadd02131c4 admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:03.321255 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb3feac-55f5-4222-a5e0-2eadd02131c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:03.321526 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb3feac-55f5-4222-a5e0-2eadd02131c4 admin admin] PATH_INFO: `/v3/users` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:03.322184 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb3feac-55f5-4222-a5e0-2eadd02131c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:31:03.324290 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb3feac-55f5-4222-a5e0-2eadd02131c4 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:31:03.353507 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 953/1927] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Nov 2 07:31:03 2022] POST /identity/v3/users => generated 294 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:31:03.354190 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3154c21a-9d01-4560-8363-c9552bf6fcfc admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:31:03.370140 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3154c21a-9d01-4560-8363-c9552bf6fcfc admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:31:03.383640 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac5ab15-5953-4a02-841f-308173d5c0cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:31:03.384342 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac5ab15-5953-4a02-841f-308173d5c0cb admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:03.384656 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac5ab15-5953-4a02-841f-308173d5c0cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:03.385134 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac5ab15-5953-4a02-841f-308173d5c0cb admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:03.386353 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac5ab15-5953-4a02-841f-308173d5c0cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:31:03.396116 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac5ab15-5953-4a02-841f-308173d5c0cb admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:31:03.411699 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 975/1928] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:31:03 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:31:03.415531 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb3feac-55f5-4222-a5e0-2eadd02131c4 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:31:03.427442 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb3feac-55f5-4222-a5e0-2eadd02131c4 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:31:03.433435 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0233491a-7269-4d19-83b0-1f1aaf1fd298 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:31:03.434386 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0233491a-7269-4d19-83b0-1f1aaf1fd298 admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:03.434807 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0233491a-7269-4d19-83b0-1f1aaf1fd298 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:03.435287 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0233491a-7269-4d19-83b0-1f1aaf1fd298 admin admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:03.436161 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0233491a-7269-4d19-83b0-1f1aaf1fd298 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:31:03.438100 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0233491a-7269-4d19-83b0-1f1aaf1fd298 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:31:03.454804 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 954/1929] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:31:03 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:31:03.455264 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac5ab15-5953-4a02-841f-308173d5c0cb admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:31:03.466164 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac5ab15-5953-4a02-841f-308173d5c0cb admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:31:03.484762 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1c53a9-acd1-4dd9-a7a1-ce054a6ab42e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:31:03.485869 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1c53a9-acd1-4dd9-a7a1-ce054a6ab42e admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:03.486212 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1c53a9-acd1-4dd9-a7a1-ce054a6ab42e admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:03.486626 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1c53a9-acd1-4dd9-a7a1-ce054a6ab42e admin admin] PATH_INFO: `/v3/projects/20a877fe29504f23b25072daff032068/users/4cdb247032804c92a7c16074ffa2b6de/roles/b494fedd00f24f0b84a529e8672a156a` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:03.520618 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1c53a9-acd1-4dd9-a7a1-ce054a6ab42e admin admin] RBAC: Authorizing `identity:create_grant(project_id=20a877fe29504f23b25072daff032068, user_id=4cdb247032804c92a7c16074ffa2b6de, role_id=b494fedd00f24f0b84a529e8672a156a)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:31:03.523704 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1c53a9-acd1-4dd9-a7a1-ce054a6ab42e admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:31:03.538791 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 976/1930] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Nov 2 07:31:03 2022] PUT /identity/v3/projects/20a877fe29504f23b25072daff032068/users/4cdb247032804c92a7c16074ffa2b6de/roles/b494fedd00f24f0b84a529e8672a156a => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 07:31:03.549626 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b702501b-7e9f-4e24-8ed5-1be5753406b0 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:03.549848 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b702501b-7e9f-4e24-8ed5-1be5753406b0 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:03.550117 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b702501b-7e9f-4e24-8ed5-1be5753406b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:03.605775 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-b702501b-7e9f-4e24-8ed5-1be5753406b0 None None] MFA Rules not processed for user `4cdb247032804c92a7c16074ffa2b6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:31:03.607820 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-b702501b-7e9f-4e24-8ed5-1be5753406b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:31:03.630316 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:31:03.708753 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 955/1931] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:31:03 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:31:03.712862 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1c53a9-acd1-4dd9-a7a1-ce054a6ab42e admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:31:03.765379 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b702501b-7e9f-4e24-8ed5-1be5753406b0 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:31:03.768074 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-b702501b-7e9f-4e24-8ed5-1be5753406b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:31:03.816756 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1c53a9-acd1-4dd9-a7a1-ce054a6ab42e admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:31:03.829665 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d19fb5-0f0b-4947-b0db-5a49e19cbc15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:31:03.831277 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d19fb5-0f0b-4947-b0db-5a49e19cbc15 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:03.831613 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d19fb5-0f0b-4947-b0db-5a49e19cbc15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:03.832037 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d19fb5-0f0b-4947-b0db-5a49e19cbc15 admin admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:03.833057 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d19fb5-0f0b-4947-b0db-5a49e19cbc15 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:31:03.835872 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d19fb5-0f0b-4947-b0db-5a49e19cbc15 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:31:03.883363 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 977/1932] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Nov 2 07:31:03 2022] POST /identity/v3/projects => generated 385 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 07:31:03.884812 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d19fb5-0f0b-4947-b0db-5a49e19cbc15 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:31:03.921273 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b702501b-7e9f-4e24-8ed5-1be5753406b0 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:31:03.977541 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d19fb5-0f0b-4947-b0db-5a49e19cbc15 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:31:03.985077 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a21911-28ae-4663-bf13-911c862e0399 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:31:03.985842 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a21911-28ae-4663-bf13-911c862e0399 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:03.986187 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a21911-28ae-4663-bf13-911c862e0399 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:03.986604 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a21911-28ae-4663-bf13-911c862e0399 admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:03.988451 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a21911-28ae-4663-bf13-911c862e0399 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:31:03.991069 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a21911-28ae-4663-bf13-911c862e0399 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:31:03.996019 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71149c2-1750-4491-b47f-fcb089e258d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:31:03.996894 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71149c2-1750-4491-b47f-fcb089e258d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:03.997351 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71149c2-1750-4491-b47f-fcb089e258d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:03.998384 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71149c2-1750-4491-b47f-fcb089e258d9 admin admin] PATH_INFO: `/v3/projects/60d8bc8a38104119870a3716af4bf2e3` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:03.999816 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71149c2-1750-4491-b47f-fcb089e258d9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=60d8bc8a38104119870a3716af4bf2e3)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:31:04.004941 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71149c2-1750-4491-b47f-fcb089e258d9 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:31:04.023863 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 978/1933] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Nov 2 07:31:03 2022] POST /identity/v3/users => generated 299 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:31:04.025577 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a21911-28ae-4663-bf13-911c862e0399 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:31:04.043835 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:31:04.051656 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a21911-28ae-4663-bf13-911c862e0399 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:31:04.059326 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2c81e2-38a4-417d-8e00-7cce93b86437 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:31:04.059940 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2c81e2-38a4-417d-8e00-7cce93b86437 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:04.060872 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2c81e2-38a4-417d-8e00-7cce93b86437 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:04.061207 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2c81e2-38a4-417d-8e00-7cce93b86437 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:04.062194 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2c81e2-38a4-417d-8e00-7cce93b86437 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:31:04.063580 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2c81e2-38a4-417d-8e00-7cce93b86437 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:31:04.078737 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-e71149c2-1750-4491-b47f-fcb089e258d9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '60d8bc8a38104119870a3716af4bf2e3', 'request_id': 'req-e71149c2-1750-4491-b47f-fcb089e258d9'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:31:04.079915 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 979/1934] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:31:04 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 07:31:04.080682 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2c81e2-38a4-417d-8e00-7cce93b86437 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:31:04.085732 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-e71149c2-1750-4491-b47f-fcb089e258d9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '60d8bc8a38104119870a3716af4bf2e3', 'request_id': 'req-e71149c2-1750-4491-b47f-fcb089e258d9'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:31:04.093429 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2c81e2-38a4-417d-8e00-7cce93b86437 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:31:04.098972 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-e71149c2-1750-4491-b47f-fcb089e258d9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '60d8bc8a38104119870a3716af4bf2e3', 'request_id': 'req-e71149c2-1750-4491-b47f-fcb089e258d9'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:31:04.109159 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:31:04.115169 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-e71149c2-1750-4491-b47f-fcb089e258d9 admin admin] The token cache is being invalidate because project 60d8bc8a38104119870a3716af4bf2e3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81501) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:31:04.115867 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-e71149c2-1750-4491-b47f-fcb089e258d9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:31:04.117572 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-e71149c2-1750-4491-b47f-fcb089e258d9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:31:04.120076 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 956/1935] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Nov 2 07:31:03 2022] DELETE /identity/v3/projects/60d8bc8a38104119870a3716af4bf2e3 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:31:04.124609 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71149c2-1750-4491-b47f-fcb089e258d9 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:31:04.126579 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-e71149c2-1750-4491-b47f-fcb089e258d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:31:04.167039 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9fa57e-806d-4dca-b286-28b3d4caff03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:31:04.167714 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9fa57e-806d-4dca-b286-28b3d4caff03 admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:04.167998 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9fa57e-806d-4dca-b286-28b3d4caff03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:04.168278 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9fa57e-806d-4dca-b286-28b3d4caff03 admin admin] PATH_INFO: `/v3/projects/2035a822d70a4ef685b052738d638027/users/80e7bef52ecb483ca9f00af3b558c29e/roles/c4bf019fc6c44c71a6c984f6d0bab75f` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:04.185949 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9fa57e-806d-4dca-b286-28b3d4caff03 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2035a822d70a4ef685b052738d638027, user_id=80e7bef52ecb483ca9f00af3b558c29e, role_id=c4bf019fc6c44c71a6c984f6d0bab75f)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:31:04.189068 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9fa57e-806d-4dca-b286-28b3d4caff03 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:31:04.206087 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 980/1936] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Nov 2 07:31:04 2022] PUT /identity/v3/projects/2035a822d70a4ef685b052738d638027/users/80e7bef52ecb483ca9f00af3b558c29e/roles/c4bf019fc6c44c71a6c984f6d0bab75f => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:31:04.210057 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9fa57e-806d-4dca-b286-28b3d4caff03 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:31:04.211819 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-0c9fa57e-806d-4dca-b286-28b3d4caff03 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:31:04.223550 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71149c2-1750-4491-b47f-fcb089e258d9 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:31:04.285656 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9fa57e-806d-4dca-b286-28b3d4caff03 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:31:04.291941 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac3c4b4-960a-4590-8261-8465e7cb1ef5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:31:04.292646 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac3c4b4-960a-4590-8261-8465e7cb1ef5 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:04.292988 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac3c4b4-960a-4590-8261-8465e7cb1ef5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:04.293378 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac3c4b4-960a-4590-8261-8465e7cb1ef5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:04.294138 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac3c4b4-960a-4590-8261-8465e7cb1ef5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:31:04.297436 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac3c4b4-960a-4590-8261-8465e7cb1ef5 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:31:04.299110 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871da87e-8232-4d93-bcc8-4826744e6f3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:31:04.299838 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871da87e-8232-4d93-bcc8-4826744e6f3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:04.300124 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871da87e-8232-4d93-bcc8-4826744e6f3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:04.300396 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871da87e-8232-4d93-bcc8-4826744e6f3f admin admin] PATH_INFO: `/v3/users/f094282c85e14e2b93c89ef1ab1a98b2` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:04.301568 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871da87e-8232-4d93-bcc8-4826744e6f3f admin admin] RBAC: Authorizing `identity:delete_user(user_id=f094282c85e14e2b93c89ef1ab1a98b2)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:31:04.303995 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871da87e-8232-4d93-bcc8-4826744e6f3f admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:31:04.337148 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 981/1937] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:31:04 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 07:31:04.338422 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac3c4b4-960a-4590-8261-8465e7cb1ef5 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:31:04.352271 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac3c4b4-960a-4590-8261-8465e7cb1ef5 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:31:04.357846 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0ec278-f575-4168-a0d4-c1a78f4f3148 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:31:04.358570 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0ec278-f575-4168-a0d4-c1a78f4f3148 admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:04.358826 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0ec278-f575-4168-a0d4-c1a78f4f3148 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:04.359095 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0ec278-f575-4168-a0d4-c1a78f4f3148 admin admin] PATH_INFO: `/v3/projects/2035a822d70a4ef685b052738d638027/users/80e7bef52ecb483ca9f00af3b558c29e/roles/c4bf019fc6c44c71a6c984f6d0bab75f` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:04.361023 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0ec278-f575-4168-a0d4-c1a78f4f3148 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2035a822d70a4ef685b052738d638027, user_id=80e7bef52ecb483ca9f00af3b558c29e, role_id=c4bf019fc6c44c71a6c984f6d0bab75f)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:31:04.363729 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0ec278-f575-4168-a0d4-c1a78f4f3148 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:31:04.377195 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-871da87e-8232-4d93-bcc8-4826744e6f3f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f094282c85e14e2b93c89ef1ab1a98b2'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:31:04.388232 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 982/1938] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Nov 2 07:31:04 2022] PUT /identity/v3/projects/2035a822d70a4ef685b052738d638027/users/80e7bef52ecb483ca9f00af3b558c29e/roles/c4bf019fc6c44c71a6c984f6d0bab75f => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 07:31:04.390890 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4979057e-76ed-400a-9c62-52ee0097ef3e None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:04.391251 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4979057e-76ed-400a-9c62-52ee0097ef3e None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:04.391533 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4979057e-76ed-400a-9c62-52ee0097ef3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:04.395388 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-871da87e-8232-4d93-bcc8-4826744e6f3f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f094282c85e14e2b93c89ef1ab1a98b2'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:31:04.396232 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-871da87e-8232-4d93-bcc8-4826744e6f3f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f094282c85e14e2b93c89ef1ab1a98b2'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:31:04.407955 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-871da87e-8232-4d93-bcc8-4826744e6f3f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f094282c85e14e2b93c89ef1ab1a98b2'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:31:04.425311 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-4979057e-76ed-400a-9c62-52ee0097ef3e None None] MFA Rules not processed for user `80e7bef52ecb483ca9f00af3b558c29e`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:31:04.426140 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-871da87e-8232-4d93-bcc8-4826744e6f3f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f094282c85e14e2b93c89ef1ab1a98b2'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:31:04.426619 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-4979057e-76ed-400a-9c62-52ee0097ef3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:31:04.438665 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 957/1939] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Nov 2 07:31:04 2022] DELETE /identity/v3/users/f094282c85e14e2b93c89ef1ab1a98b2 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 07:31:04.479285 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:31:04.510309 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 983/1940] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:31:04 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:31:04.522608 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871da87e-8232-4d93-bcc8-4826744e6f3f admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:31:04.524424 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-871da87e-8232-4d93-bcc8-4826744e6f3f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:31:04.558574 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:31:04.636835 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871da87e-8232-4d93-bcc8-4826744e6f3f admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:31:04.661157 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23d2b8f-c693-4ccd-842c-9dc2a85319d1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43bb326fb82e4d2c83fb2eaf91860dc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:31:04.661816 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23d2b8f-c693-4ccd-842c-9dc2a85319d1 service cinder] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:04.662102 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23d2b8f-c693-4ccd-842c-9dc2a85319d1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:04.662410 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23d2b8f-c693-4ccd-842c-9dc2a85319d1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:04.743596 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23d2b8f-c693-4ccd-842c-9dc2a85319d1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:31:04.745411 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23d2b8f-c693-4ccd-842c-9dc2a85319d1 service cinder] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:31:04.756804 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 958/1941] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:31:04 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:31:04.812031 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4979057e-76ed-400a-9c62-52ee0097ef3e None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:31:04.814018 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-4979057e-76ed-400a-9c62-52ee0097ef3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:31:04.898163 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4979057e-76ed-400a-9c62-52ee0097ef3e None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:31:04.955736 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36179f00-e468-4632-82ee-421564433e43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:31:04.956259 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36179f00-e468-4632-82ee-421564433e43 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:04.956590 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36179f00-e468-4632-82ee-421564433e43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:04.957069 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36179f00-e468-4632-82ee-421564433e43 admin admin] PATH_INFO: `/v3/projects/32ad260aef134e49851252063e7cf960` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:04.958341 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36179f00-e468-4632-82ee-421564433e43 admin admin] RBAC: Authorizing `identity:delete_project(project_id=32ad260aef134e49851252063e7cf960)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:31:04.968155 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36179f00-e468-4632-82ee-421564433e43 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:31:05.019988 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-36179f00-e468-4632-82ee-421564433e43 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '32ad260aef134e49851252063e7cf960', 'request_id': 'req-36179f00-e468-4632-82ee-421564433e43'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:31:05.026323 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-36179f00-e468-4632-82ee-421564433e43 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '32ad260aef134e49851252063e7cf960', 'request_id': 'req-36179f00-e468-4632-82ee-421564433e43'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:31:05.032391 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-36179f00-e468-4632-82ee-421564433e43 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '32ad260aef134e49851252063e7cf960', 'request_id': 'req-36179f00-e468-4632-82ee-421564433e43'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:31:05.046284 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-36179f00-e468-4632-82ee-421564433e43 admin admin] The token cache is being invalidate because project 32ad260aef134e49851252063e7cf960 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:31:05.046616 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-36179f00-e468-4632-82ee-421564433e43 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:31:05.047276 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-36179f00-e468-4632-82ee-421564433e43 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:31:05.048812 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 984/1942] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Nov 2 07:31:04 2022] DELETE /identity/v3/projects/32ad260aef134e49851252063e7cf960 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:31:05.053745 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31695a67-618b-4bef-b8b2-dfcb9639c645 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:05.054027 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31695a67-618b-4bef-b8b2-dfcb9639c645 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:05.054293 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31695a67-618b-4bef-b8b2-dfcb9639c645 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:05.076402 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-31695a67-618b-4bef-b8b2-dfcb9639c645 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:31:05.077820 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-31695a67-618b-4bef-b8b2-dfcb9639c645 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:31:05.146263 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 959/1943] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:31:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:31:05.155204 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95007121-edf1-47b0-a43a-fab4d92c2826 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:05.155598 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95007121-edf1-47b0-a43a-fab4d92c2826 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:05.155952 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95007121-edf1-47b0-a43a-fab4d92c2826 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:05.190616 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-95007121-edf1-47b0-a43a-fab4d92c2826 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:31:05.191820 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-95007121-edf1-47b0-a43a-fab4d92c2826 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:31:05.204839 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 985/1944] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:31:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:31:05.207032 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31695a67-618b-4bef-b8b2-dfcb9639c645 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:31:05.218622 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31695a67-618b-4bef-b8b2-dfcb9639c645 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:31:05.226719 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc263d9-1697-4c88-ba98-7f24f253dd44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:31:05.227328 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc263d9-1697-4c88-ba98-7f24f253dd44 admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:05.227606 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc263d9-1697-4c88-ba98-7f24f253dd44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:05.227953 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc263d9-1697-4c88-ba98-7f24f253dd44 admin admin] PATH_INFO: `/v3/domains` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:05.228509 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc263d9-1697-4c88-ba98-7f24f253dd44 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:31:05.228782 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc263d9-1697-4c88-ba98-7f24f253dd44 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:31:05.230593 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc263d9-1697-4c88-ba98-7f24f253dd44 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:31:05.246678 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 960/1945] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Nov 2 07:31:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:31:05.248994 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95007121-edf1-47b0-a43a-fab4d92c2826 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:31:05.258768 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95007121-edf1-47b0-a43a-fab4d92c2826 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:31:05.271835 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f1893f-ea95-4b20-96c0-cd0b85652132 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:31:05.272456 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f1893f-ea95-4b20-96c0-cd0b85652132 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:05.272732 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f1893f-ea95-4b20-96c0-cd0b85652132 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:05.273000 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f1893f-ea95-4b20-96c0-cd0b85652132 admin admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:05.274231 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f1893f-ea95-4b20-96c0-cd0b85652132 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:31:05.276355 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f1893f-ea95-4b20-96c0-cd0b85652132 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:31:05.292624 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:31:05.320849 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 986/1946] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Nov 2 07:31:05 2022] POST /identity/v3/projects => generated 373 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:31:05.322001 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc263d9-1697-4c88-ba98-7f24f253dd44 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:31:05.333338 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc263d9-1697-4c88-ba98-7f24f253dd44 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:31:05.339235 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f1893f-ea95-4b20-96c0-cd0b85652132 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:31:05.341098 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-76f1893f-ea95-4b20-96c0-cd0b85652132 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:31:05.399976 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a4234f-8850-447c-ae5b-e22507629fc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:31:05.402742 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a4234f-8850-447c-ae5b-e22507629fc7 admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:05.403029 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a4234f-8850-447c-ae5b-e22507629fc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:05.403357 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a4234f-8850-447c-ae5b-e22507629fc7 admin admin] PATH_INFO: `/v3/users` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:05.405582 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a4234f-8850-447c-ae5b-e22507629fc7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:31:05.407565 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a4234f-8850-447c-ae5b-e22507629fc7 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:31:05.419214 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:31:05.434322 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f1893f-ea95-4b20-96c0-cd0b85652132 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:31:05.438702 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 961/1947] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Nov 2 07:31:05 2022] POST /identity/v3/users => generated 287 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 07:31:05.440284 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4273a8e-4356-4e0e-951d-647b2d775f76 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43bb326fb82e4d2c83fb2eaf91860dc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:31:05.441219 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4273a8e-4356-4e0e-951d-647b2d775f76 service cinder] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:05.441641 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4273a8e-4356-4e0e-951d-647b2d775f76 service cinder] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:05.442060 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4273a8e-4356-4e0e-951d-647b2d775f76 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:05.443854 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-d4273a8e-4356-4e0e-951d-647b2d775f76 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:31:05.451974 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a4234f-8850-447c-ae5b-e22507629fc7 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:31:05.462180 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a4234f-8850-447c-ae5b-e22507629fc7 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:31:05.467695 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09489324-ad1b-4538-a261-ae4d6dcf8269 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:31:05.468356 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09489324-ad1b-4538-a261-ae4d6dcf8269 admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:05.468706 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09489324-ad1b-4538-a261-ae4d6dcf8269 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:05.469045 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09489324-ad1b-4538-a261-ae4d6dcf8269 admin admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:05.469746 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09489324-ad1b-4538-a261-ae4d6dcf8269 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:31:05.471564 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09489324-ad1b-4538-a261-ae4d6dcf8269 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:31:05.482504 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 962/1948] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:31:05 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:31:05.486332 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09489324-ad1b-4538-a261-ae4d6dcf8269 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:31:05.496170 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09489324-ad1b-4538-a261-ae4d6dcf8269 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:31:05.501661 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be20243-41c8-420e-86ac-d6959b990714 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:31:05.502381 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be20243-41c8-420e-86ac-d6959b990714 admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:05.502838 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be20243-41c8-420e-86ac-d6959b990714 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:05.503120 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be20243-41c8-420e-86ac-d6959b990714 admin admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:05.503743 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be20243-41c8-420e-86ac-d6959b990714 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:31:05.505483 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be20243-41c8-420e-86ac-d6959b990714 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:31:05.506397 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4273a8e-4356-4e0e-951d-647b2d775f76 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:31:05.507826 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4273a8e-4356-4e0e-951d-647b2d775f76 service cinder] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:31:05.519287 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 963/1949] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:31:05 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 07:31:05.519448 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 987/1950] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:31:05 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 07:31:05.520663 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be20243-41c8-420e-86ac-d6959b990714 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:31:05.531432 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be20243-41c8-420e-86ac-d6959b990714 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:31:05.536210 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fba6c1-c013-4c50-84a2-5188490bbfee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:31:05.536853 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fba6c1-c013-4c50-84a2-5188490bbfee admin admin] REQUEST_METHOD: `PUT` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:05.537131 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fba6c1-c013-4c50-84a2-5188490bbfee admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:05.537399 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fba6c1-c013-4c50-84a2-5188490bbfee admin admin] PATH_INFO: `/v3/projects/534d58cf08874a04883209846b71d662/users/88b76f195ba54501a3120b52479597b2/roles/b494fedd00f24f0b84a529e8672a156a` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:05.552061 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fba6c1-c013-4c50-84a2-5188490bbfee admin admin] RBAC: Authorizing `identity:create_grant(project_id=534d58cf08874a04883209846b71d662, user_id=88b76f195ba54501a3120b52479597b2, role_id=b494fedd00f24f0b84a529e8672a156a)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:31:05.554874 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fba6c1-c013-4c50-84a2-5188490bbfee admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:31:05.569706 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 964/1951] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Nov 2 07:31:05 2022] PUT /identity/v3/projects/534d58cf08874a04883209846b71d662/users/88b76f195ba54501a3120b52479597b2/roles/b494fedd00f24f0b84a529e8672a156a => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 07:31:05.582587 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896a37a2-0e16-465f-a258-91a6906a09fc None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:05.583163 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896a37a2-0e16-465f-a258-91a6906a09fc None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:05.583506 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896a37a2-0e16-465f-a258-91a6906a09fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:05.613985 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-896a37a2-0e16-465f-a258-91a6906a09fc None None] MFA Rules not processed for user `88b76f195ba54501a3120b52479597b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:31:05.615228 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-896a37a2-0e16-465f-a258-91a6906a09fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:31:05.699525 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 988/1952] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:31:05 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:31:05.711029 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fba6c1-c013-4c50-84a2-5188490bbfee admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:31:05.712824 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-78fba6c1-c013-4c50-84a2-5188490bbfee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:31:05.791254 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fba6c1-c013-4c50-84a2-5188490bbfee admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:31:05.795592 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3eda6a8-4d88-4e72-bc70-d9dec0b2bf51 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43bb326fb82e4d2c83fb2eaf91860dc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:31:05.796204 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3eda6a8-4d88-4e72-bc70-d9dec0b2bf51 service cinder] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:05.796488 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3eda6a8-4d88-4e72-bc70-d9dec0b2bf51 service cinder] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:05.796763 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3eda6a8-4d88-4e72-bc70-d9dec0b2bf51 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:05.806570 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3eda6a8-4d88-4e72-bc70-d9dec0b2bf51 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:31:05.808068 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3eda6a8-4d88-4e72-bc70-d9dec0b2bf51 service cinder] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:31:05.818304 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 965/1953] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:31:05 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:31:05.858846 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896a37a2-0e16-465f-a258-91a6906a09fc None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:31:05.909008 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:31:05.938230 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896a37a2-0e16-465f-a258-91a6906a09fc None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:31:05.944974 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b95c29-9d9a-480c-a336-2b3e8c876400 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:31:05.944974 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b95c29-9d9a-480c-a336-2b3e8c876400 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:05.944974 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b95c29-9d9a-480c-a336-2b3e8c876400 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:05.944974 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b95c29-9d9a-480c-a336-2b3e8c876400 admin admin] PATH_INFO: `/v3/users/88b76f195ba54501a3120b52479597b2` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:05.946127 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b95c29-9d9a-480c-a336-2b3e8c876400 admin admin] RBAC: Authorizing `identity:delete_user(user_id=88b76f195ba54501a3120b52479597b2)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:31:05.948059 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b95c29-9d9a-480c-a336-2b3e8c876400 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:31:06.000490 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-96b95c29-9d9a-480c-a336-2b3e8c876400 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '88b76f195ba54501a3120b52479597b2'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:31:06.008145 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-96b95c29-9d9a-480c-a336-2b3e8c876400 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '88b76f195ba54501a3120b52479597b2'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:31:06.008864 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-96b95c29-9d9a-480c-a336-2b3e8c876400 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '88b76f195ba54501a3120b52479597b2'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:31:06.016146 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-96b95c29-9d9a-480c-a336-2b3e8c876400 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '88b76f195ba54501a3120b52479597b2'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:31:06.016798 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-96b95c29-9d9a-480c-a336-2b3e8c876400 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '88b76f195ba54501a3120b52479597b2'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:31:06.034875 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 989/1954] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Nov 2 07:31:05 2022] DELETE /identity/v3/users/88b76f195ba54501a3120b52479597b2 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:31:06.039492 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa4473f-c9e3-44fa-8ad9-b73e8c2adfab None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:06.039900 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa4473f-c9e3-44fa-8ad9-b73e8c2adfab None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:06.040524 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa4473f-c9e3-44fa-8ad9-b73e8c2adfab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:06.070818 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-5aa4473f-c9e3-44fa-8ad9-b73e8c2adfab None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:31:06.071871 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-5aa4473f-c9e3-44fa-8ad9-b73e8c2adfab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:31:06.132087 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 966/1955] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:31:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:31:06.143202 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b95c29-9d9a-480c-a336-2b3e8c876400 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:31:06.144588 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-96b95c29-9d9a-480c-a336-2b3e8c876400 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:31:06.200690 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b95c29-9d9a-480c-a336-2b3e8c876400 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:31:06.205298 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6033cd-c394-4e86-9703-2d6d321ff1a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3ec6ecb85f94ef59762877d9ea33ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:31:06.205902 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6033cd-c394-4e86-9703-2d6d321ff1a5 service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:06.206182 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6033cd-c394-4e86-9703-2d6d321ff1a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:06.206488 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6033cd-c394-4e86-9703-2d6d321ff1a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:06.215991 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6033cd-c394-4e86-9703-2d6d321ff1a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:31:06.217546 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6033cd-c394-4e86-9703-2d6d321ff1a5 service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:31:06.227406 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 990/1956] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:31:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:31:06.557057 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa4473f-c9e3-44fa-8ad9-b73e8c2adfab None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:31:06.558593 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-5aa4473f-c9e3-44fa-8ad9-b73e8c2adfab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:31:06.570208 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa4473f-c9e3-44fa-8ad9-b73e8c2adfab None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:31:06.577043 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fa9392-b17a-4f44-a298-51c88f561de1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:31:06.577735 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fa9392-b17a-4f44-a298-51c88f561de1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:06.578053 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fa9392-b17a-4f44-a298-51c88f561de1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:06.578337 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fa9392-b17a-4f44-a298-51c88f561de1 admin admin] PATH_INFO: `/v3/projects/534d58cf08874a04883209846b71d662` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:06.579520 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fa9392-b17a-4f44-a298-51c88f561de1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=534d58cf08874a04883209846b71d662)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:31:06.581529 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fa9392-b17a-4f44-a298-51c88f561de1 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:31:06.637116 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:INSERT=3,keystone:SELECT=94,keystone:DELETE=3 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:31:06.642873 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-b1fa9392-b17a-4f44-a298-51c88f561de1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '534d58cf08874a04883209846b71d662', 'request_id': 'req-b1fa9392-b17a-4f44-a298-51c88f561de1'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:31:06.650590 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-b1fa9392-b17a-4f44-a298-51c88f561de1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '534d58cf08874a04883209846b71d662', 'request_id': 'req-b1fa9392-b17a-4f44-a298-51c88f561de1'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:31:06.658475 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-b1fa9392-b17a-4f44-a298-51c88f561de1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '534d58cf08874a04883209846b71d662', 'request_id': 'req-b1fa9392-b17a-4f44-a298-51c88f561de1'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:31:06.668760 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-b1fa9392-b17a-4f44-a298-51c88f561de1 admin admin] The token cache is being invalidate because project 534d58cf08874a04883209846b71d662 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81501) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:31:06.669050 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-b1fa9392-b17a-4f44-a298-51c88f561de1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:31:06.670880 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-b1fa9392-b17a-4f44-a298-51c88f561de1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:31:06.672419 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 967/1957] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Nov 2 07:31:06 2022] DELETE /identity/v3/projects/534d58cf08874a04883209846b71d662 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:31:08.934549 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a399f64-0ac8-4999-9eaa-2f0b5276a786 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:08.935055 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a399f64-0ac8-4999-9eaa-2f0b5276a786 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:08.935220 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a399f64-0ac8-4999-9eaa-2f0b5276a786 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:08.954980 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-4a399f64-0ac8-4999-9eaa-2f0b5276a786 None None] MFA Rules not processed for user `ff4736f3d50746b0bf5d6a917b2ca0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:31:08.956139 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-4a399f64-0ac8-4999-9eaa-2f0b5276a786 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:31:09.011213 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 991/1958] 10.222.0.18 () {58 vars in 1009 bytes} [Wed Nov 2 07:31:08 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:31:09.064388 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fa9392-b17a-4f44-a298-51c88f561de1 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:31:09.065840 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-b1fa9392-b17a-4f44-a298-51c88f561de1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:31:09.153122 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fa9392-b17a-4f44-a298-51c88f561de1 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:31:09.160180 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293e7818-fd14-4698-baff-38c00fd5db40 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9116f47889f4db39e0203d6cdae1593', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:31:09.161245 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293e7818-fd14-4698-baff-38c00fd5db40 service swift] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:09.161741 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293e7818-fd14-4698-baff-38c00fd5db40 service swift] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:09.162227 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293e7818-fd14-4698-baff-38c00fd5db40 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:09.180249 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293e7818-fd14-4698-baff-38c00fd5db40 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:31:09.183873 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293e7818-fd14-4698-baff-38c00fd5db40 service swift] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:31:09.198509 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 968/1959] 10.222.0.18 () {60 vars in 1384 bytes} [Wed Nov 2 07:31:09 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Nov 02 07:31:09.760201 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d357a5-4fae-4b58-83cf-4afc7e8acb3e None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:09.760543 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d357a5-4fae-4b58-83cf-4afc7e8acb3e None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:09.760827 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d357a5-4fae-4b58-83cf-4afc7e8acb3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:09.768427 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:31:09.791412 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-c3d357a5-4fae-4b58-83cf-4afc7e8acb3e None None] MFA Rules not processed for user `fe26aea6bb9d4dcebf046da719b2c244`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:31:09.792683 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-c3d357a5-4fae-4b58-83cf-4afc7e8acb3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:31:09.824896 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd05523-07d4-446c-9d39-ae1f89ccaa67 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:09.825180 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd05523-07d4-446c-9d39-ae1f89ccaa67 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:09.825453 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd05523-07d4-446c-9d39-ae1f89ccaa67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:09.854030 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-ccd05523-07d4-446c-9d39-ae1f89ccaa67 None None] MFA Rules not processed for user `ff4736f3d50746b0bf5d6a917b2ca0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:31:09.855937 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-ccd05523-07d4-446c-9d39-ae1f89ccaa67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:31:09.868428 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 969/1960] 10.222.0.18 () {58 vars in 967 bytes} [Wed Nov 2 07:31:09 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:31:09.876040 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 992/1961] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:31:09 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:31:09.886511 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd05523-07d4-446c-9d39-ae1f89ccaa67 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:31:09.899367 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd05523-07d4-446c-9d39-ae1f89ccaa67 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:31:09.905129 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6fcd94-63f7-4228-ad89-f2f893195557 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9116f47889f4db39e0203d6cdae1593', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:31:09.905808 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6fcd94-63f7-4228-ad89-f2f893195557 service swift] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:09.906738 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6fcd94-63f7-4228-ad89-f2f893195557 service swift] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:09.906738 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6fcd94-63f7-4228-ad89-f2f893195557 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:09.916810 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6fcd94-63f7-4228-ad89-f2f893195557 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:31:09.919015 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6fcd94-63f7-4228-ad89-f2f893195557 service swift] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:31:09.928224 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 970/1962] 10.222.0.18 () {60 vars in 1384 bytes} [Wed Nov 2 07:31:09 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Nov 02 07:31:10.183470 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac1099b-2fff-466e-9a5e-4b6a5f203801 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:10.183953 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac1099b-2fff-466e-9a5e-4b6a5f203801 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:10.184238 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac1099b-2fff-466e-9a5e-4b6a5f203801 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:10.207654 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-6ac1099b-2fff-466e-9a5e-4b6a5f203801 None None] MFA Rules not processed for user `ff4736f3d50746b0bf5d6a917b2ca0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:31:10.209075 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-6ac1099b-2fff-466e-9a5e-4b6a5f203801 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:31:10.218972 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 993/1963] 10.222.0.18 () {58 vars in 1009 bytes} [Wed Nov 2 07:31:10 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:31:10.231054 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6fcd94-63f7-4228-ad89-f2f893195557 service swift] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:31:10.242596 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6fcd94-63f7-4228-ad89-f2f893195557 service swift] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:31:10.248639 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdfadd0-cae3-479e-97fd-442fcd660240 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9116f47889f4db39e0203d6cdae1593', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:31:10.249385 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdfadd0-cae3-479e-97fd-442fcd660240 service swift] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:10.249671 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdfadd0-cae3-479e-97fd-442fcd660240 service swift] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:10.249941 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdfadd0-cae3-479e-97fd-442fcd660240 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:10.258696 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdfadd0-cae3-479e-97fd-442fcd660240 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:31:10.260373 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdfadd0-cae3-479e-97fd-442fcd660240 service swift] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:31:10.270491 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 971/1964] 10.222.0.18 () {60 vars in 1384 bytes} [Wed Nov 2 07:31:10 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Nov 02 07:31:10.754929 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a469e860-5158-4e0a-bcf9-73029aea61f1 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:10.755399 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a469e860-5158-4e0a-bcf9-73029aea61f1 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:10.755802 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a469e860-5158-4e0a-bcf9-73029aea61f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:10.778317 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-a469e860-5158-4e0a-bcf9-73029aea61f1 None None] MFA Rules not processed for user `ff4736f3d50746b0bf5d6a917b2ca0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:31:10.780964 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-a469e860-5158-4e0a-bcf9-73029aea61f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:31:10.789524 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 994/1965] 10.222.0.18 () {58 vars in 967 bytes} [Wed Nov 2 07:31:10 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:31:10.805638 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdfadd0-cae3-479e-97fd-442fcd660240 service swift] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:31:10.820688 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdfadd0-cae3-479e-97fd-442fcd660240 service swift] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:31:10.829898 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd23c37-4f83-45f9-b20e-f33161be2b25 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9116f47889f4db39e0203d6cdae1593', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:31:10.830633 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd23c37-4f83-45f9-b20e-f33161be2b25 service swift] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:10.831418 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd23c37-4f83-45f9-b20e-f33161be2b25 service swift] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:10.831889 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd23c37-4f83-45f9-b20e-f33161be2b25 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:10.843720 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd23c37-4f83-45f9-b20e-f33161be2b25 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:31:10.845447 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd23c37-4f83-45f9-b20e-f33161be2b25 service swift] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:31:10.848590 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a469e860-5158-4e0a-bcf9-73029aea61f1 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:31:10.850076 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-a469e860-5158-4e0a-bcf9-73029aea61f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:31:10.854677 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 972/1966] 10.222.0.18 () {60 vars in 1384 bytes} [Wed Nov 2 07:31:10 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Nov 02 07:31:10.922176 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a469e860-5158-4e0a-bcf9-73029aea61f1 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:31:10.926604 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2853fa1-b2f6-4f04-8f47-5a5ba099b84a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43bb326fb82e4d2c83fb2eaf91860dc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:31:10.927222 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2853fa1-b2f6-4f04-8f47-5a5ba099b84a service cinder] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:10.927499 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2853fa1-b2f6-4f04-8f47-5a5ba099b84a service cinder] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:10.927953 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2853fa1-b2f6-4f04-8f47-5a5ba099b84a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:10.930074 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-e2853fa1-b2f6-4f04-8f47-5a5ba099b84a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:31:11.007090 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2853fa1-b2f6-4f04-8f47-5a5ba099b84a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:31:11.008532 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2853fa1-b2f6-4f04-8f47-5a5ba099b84a service cinder] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:31:11.019040 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 995/1967] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:31:10 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 07:31:11.868496 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd23c37-4f83-45f9-b20e-f33161be2b25 service swift] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:31:11.870765 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-5bd23c37-4f83-45f9-b20e-f33161be2b25 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:31:11.998167 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd23c37-4f83-45f9-b20e-f33161be2b25 service swift] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:31:12.015800 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b706ee38-8b5d-4d61-8ebb-d4c6129455d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:31:12.016529 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b706ee38-8b5d-4d61-8ebb-d4c6129455d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:12.016835 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b706ee38-8b5d-4d61-8ebb-d4c6129455d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:12.017112 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b706ee38-8b5d-4d61-8ebb-d4c6129455d8 admin admin] PATH_INFO: `/v3/users/9978c2003c684acab802c2d2b29b19ff` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:12.018413 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b706ee38-8b5d-4d61-8ebb-d4c6129455d8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9978c2003c684acab802c2d2b29b19ff)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:31:12.020381 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b706ee38-8b5d-4d61-8ebb-d4c6129455d8 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:31:12.041679 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:31:12.074530 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-b706ee38-8b5d-4d61-8ebb-d4c6129455d8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9978c2003c684acab802c2d2b29b19ff'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:31:12.084280 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-b706ee38-8b5d-4d61-8ebb-d4c6129455d8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9978c2003c684acab802c2d2b29b19ff'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:31:12.085039 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-b706ee38-8b5d-4d61-8ebb-d4c6129455d8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9978c2003c684acab802c2d2b29b19ff'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:31:12.093947 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-b706ee38-8b5d-4d61-8ebb-d4c6129455d8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9978c2003c684acab802c2d2b29b19ff'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:31:12.105946 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-b706ee38-8b5d-4d61-8ebb-d4c6129455d8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9978c2003c684acab802c2d2b29b19ff'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:31:12.112157 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 973/1968] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Nov 2 07:31:11 2022] DELETE /identity/v3/users/9978c2003c684acab802c2d2b29b19ff => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:31:12.116458 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a45b1b-8f7e-45b1-8b52-0636aa3441f0 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:12.116913 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a45b1b-8f7e-45b1-8b52-0636aa3441f0 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:12.117040 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a45b1b-8f7e-45b1-8b52-0636aa3441f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:12.142245 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-c4a45b1b-8f7e-45b1-8b52-0636aa3441f0 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:31:12.144574 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-c4a45b1b-8f7e-45b1-8b52-0636aa3441f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:31:12.198573 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:31:12.218191 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 996/1969] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:31:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:31:12.231037 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b706ee38-8b5d-4d61-8ebb-d4c6129455d8 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:31:12.234656 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-b706ee38-8b5d-4d61-8ebb-d4c6129455d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:31:12.342961 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b706ee38-8b5d-4d61-8ebb-d4c6129455d8 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:31:12.347690 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f832b903-ece4-4ab1-b91e-0f4871905eba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3ec6ecb85f94ef59762877d9ea33ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:31:12.348439 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f832b903-ece4-4ab1-b91e-0f4871905eba service neutron] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:12.348830 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f832b903-ece4-4ab1-b91e-0f4871905eba service neutron] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:12.349177 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f832b903-ece4-4ab1-b91e-0f4871905eba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:12.364424 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f832b903-ece4-4ab1-b91e-0f4871905eba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:31:12.366018 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f832b903-ece4-4ab1-b91e-0f4871905eba service neutron] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:31:12.380291 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 974/1970] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:31:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:31:12.790113 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a45b1b-8f7e-45b1-8b52-0636aa3441f0 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:31:12.791655 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-c4a45b1b-8f7e-45b1-8b52-0636aa3441f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:31:12.804859 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a45b1b-8f7e-45b1-8b52-0636aa3441f0 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:31:12.810534 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c798b83f-dd71-4b9d-83bc-826a2ef149a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:31:12.811270 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c798b83f-dd71-4b9d-83bc-826a2ef149a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:12.811572 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c798b83f-dd71-4b9d-83bc-826a2ef149a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:12.811827 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c798b83f-dd71-4b9d-83bc-826a2ef149a3 admin admin] PATH_INFO: `/v3/projects/3040390b4c004b9faae74c42e65d4660` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:12.813107 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c798b83f-dd71-4b9d-83bc-826a2ef149a3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3040390b4c004b9faae74c42e65d4660)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:31:12.815106 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c798b83f-dd71-4b9d-83bc-826a2ef149a3 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:31:12.881216 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-c798b83f-dd71-4b9d-83bc-826a2ef149a3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3040390b4c004b9faae74c42e65d4660', 'request_id': 'req-c798b83f-dd71-4b9d-83bc-826a2ef149a3'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:31:12.887675 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-c798b83f-dd71-4b9d-83bc-826a2ef149a3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3040390b4c004b9faae74c42e65d4660', 'request_id': 'req-c798b83f-dd71-4b9d-83bc-826a2ef149a3'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:31:12.902445 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-c798b83f-dd71-4b9d-83bc-826a2ef149a3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3040390b4c004b9faae74c42e65d4660', 'request_id': 'req-c798b83f-dd71-4b9d-83bc-826a2ef149a3'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:31:12.920260 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-c798b83f-dd71-4b9d-83bc-826a2ef149a3 admin admin] The token cache is being invalidate because project 3040390b4c004b9faae74c42e65d4660 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:31:12.920868 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-c798b83f-dd71-4b9d-83bc-826a2ef149a3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:31:12.922763 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-c798b83f-dd71-4b9d-83bc-826a2ef149a3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:31:12.925469 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 997/1971] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Nov 2 07:31:12 2022] DELETE /identity/v3/projects/3040390b4c004b9faae74c42e65d4660 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:31:12.930281 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66630e39-51b7-4583-8d5e-7df8e8914e18 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:12.930783 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66630e39-51b7-4583-8d5e-7df8e8914e18 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:12.931054 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66630e39-51b7-4583-8d5e-7df8e8914e18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:12.960828 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-66630e39-51b7-4583-8d5e-7df8e8914e18 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:31:12.962599 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-66630e39-51b7-4583-8d5e-7df8e8914e18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:31:13.036218 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 975/1972] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:31:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:31:13.046827 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d722afc-5bcf-4365-8e4c-052ec204c07d None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:13.047117 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d722afc-5bcf-4365-8e4c-052ec204c07d None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:13.047402 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d722afc-5bcf-4365-8e4c-052ec204c07d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:13.074227 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-8d722afc-5bcf-4365-8e4c-052ec204c07d None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:31:13.075843 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-8d722afc-5bcf-4365-8e4c-052ec204c07d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:31:13.084344 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 998/1973] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:31:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:31:13.088635 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66630e39-51b7-4583-8d5e-7df8e8914e18 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:31:13.098951 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66630e39-51b7-4583-8d5e-7df8e8914e18 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:31:13.105155 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2ab7b1-ea57-41f0-9178-7ccaa1f1e8e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:31:13.105780 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2ab7b1-ea57-41f0-9178-7ccaa1f1e8e2 admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:13.106057 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2ab7b1-ea57-41f0-9178-7ccaa1f1e8e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:13.106325 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2ab7b1-ea57-41f0-9178-7ccaa1f1e8e2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:13.107655 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2ab7b1-ea57-41f0-9178-7ccaa1f1e8e2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:31:13.107960 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2ab7b1-ea57-41f0-9178-7ccaa1f1e8e2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:31:13.109555 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2ab7b1-ea57-41f0-9178-7ccaa1f1e8e2 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:31:13.128486 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 976/1974] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Nov 2 07:31:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:31:13.129038 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d722afc-5bcf-4365-8e4c-052ec204c07d None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:31:13.138945 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d722afc-5bcf-4365-8e4c-052ec204c07d None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:31:13.144797 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc463b9-85ea-4069-a37a-9c099b6076e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:31:13.145590 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc463b9-85ea-4069-a37a-9c099b6076e7 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:13.146000 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc463b9-85ea-4069-a37a-9c099b6076e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:13.146906 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc463b9-85ea-4069-a37a-9c099b6076e7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:13.147682 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc463b9-85ea-4069-a37a-9c099b6076e7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:31:13.149621 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc463b9-85ea-4069-a37a-9c099b6076e7 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:31:13.180039 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 999/1975] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Nov 2 07:31:13 2022] POST /identity/v3/projects => generated 371 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:31:13.184219 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2ab7b1-ea57-41f0-9178-7ccaa1f1e8e2 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:31:13.195225 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2ab7b1-ea57-41f0-9178-7ccaa1f1e8e2 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:31:13.242957 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:31:13.293360 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284b5810-250a-4834-a990-470278fe9b82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:31:13.294074 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284b5810-250a-4834-a990-470278fe9b82 admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:13.295049 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284b5810-250a-4834-a990-470278fe9b82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:13.295472 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284b5810-250a-4834-a990-470278fe9b82 admin admin] PATH_INFO: `/v3/users` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:13.296494 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284b5810-250a-4834-a990-470278fe9b82 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:31:13.298808 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284b5810-250a-4834-a990-470278fe9b82 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:31:13.327277 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 977/1976] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Nov 2 07:31:13 2022] POST /identity/v3/users => generated 286 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:31:13.331362 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc463b9-85ea-4069-a37a-9c099b6076e7 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:31:13.342769 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc463b9-85ea-4069-a37a-9c099b6076e7 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:31:13.348199 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13042f5-bfad-4443-bfdd-1b15b02bce2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:31:13.349128 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13042f5-bfad-4443-bfdd-1b15b02bce2f admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:13.349621 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13042f5-bfad-4443-bfdd-1b15b02bce2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:13.350095 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13042f5-bfad-4443-bfdd-1b15b02bce2f admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:13.351121 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13042f5-bfad-4443-bfdd-1b15b02bce2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:31:13.353170 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13042f5-bfad-4443-bfdd-1b15b02bce2f admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:31:13.366643 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1000/1977] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:31:13 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:31:13.370884 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284b5810-250a-4834-a990-470278fe9b82 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:31:13.387034 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284b5810-250a-4834-a990-470278fe9b82 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:31:13.394284 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3c2c11-731a-4296-b387-8ab2824786c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:31:13.395273 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3c2c11-731a-4296-b387-8ab2824786c6 admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:13.395756 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3c2c11-731a-4296-b387-8ab2824786c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:13.396227 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3c2c11-731a-4296-b387-8ab2824786c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:13.397098 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3c2c11-731a-4296-b387-8ab2824786c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:31:13.399310 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3c2c11-731a-4296-b387-8ab2824786c6 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:31:13.413492 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 978/1978] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:31:13 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:31:13.417641 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13042f5-bfad-4443-bfdd-1b15b02bce2f admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:31:13.431185 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13042f5-bfad-4443-bfdd-1b15b02bce2f admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:31:13.440530 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d3821b-66f0-415d-b1b5-a85496afd95f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:31:13.441443 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d3821b-66f0-415d-b1b5-a85496afd95f admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:13.441904 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d3821b-66f0-415d-b1b5-a85496afd95f admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:13.442425 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d3821b-66f0-415d-b1b5-a85496afd95f admin admin] PATH_INFO: `/v3/projects/155356fb94b74eaba3a8465d1a1f21ff/users/0b55941561934c818a2b5eb2c9a19d22/roles/b494fedd00f24f0b84a529e8672a156a` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:13.461624 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d3821b-66f0-415d-b1b5-a85496afd95f admin admin] RBAC: Authorizing `identity:create_grant(project_id=155356fb94b74eaba3a8465d1a1f21ff, user_id=0b55941561934c818a2b5eb2c9a19d22, role_id=b494fedd00f24f0b84a529e8672a156a)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:31:13.466701 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d3821b-66f0-415d-b1b5-a85496afd95f admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:31:13.489702 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1001/1979] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Nov 2 07:31:13 2022] PUT /identity/v3/projects/155356fb94b74eaba3a8465d1a1f21ff/users/0b55941561934c818a2b5eb2c9a19d22/roles/b494fedd00f24f0b84a529e8672a156a => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:31:13.491099 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4a5229-616b-4315-a809-ee1ee54073b8 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:13.491384 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4a5229-616b-4315-a809-ee1ee54073b8 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:13.491660 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4a5229-616b-4315-a809-ee1ee54073b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:13.517294 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-bb4a5229-616b-4315-a809-ee1ee54073b8 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:31:13.517514 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d3821b-66f0-415d-b1b5-a85496afd95f admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:31:13.517908 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-bb4a5229-616b-4315-a809-ee1ee54073b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:31:13.519236 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-20d3821b-66f0-415d-b1b5-a85496afd95f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:31:13.608717 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 979/1980] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:31:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:31:13.609637 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4a5229-616b-4315-a809-ee1ee54073b8 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:31:13.611335 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-bb4a5229-616b-4315-a809-ee1ee54073b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:31:13.625701 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4a5229-616b-4315-a809-ee1ee54073b8 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:31:13.631844 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee42a29-b28c-482a-bf21-1f04cb889ebc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:31:13.632484 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee42a29-b28c-482a-bf21-1f04cb889ebc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:13.632762 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee42a29-b28c-482a-bf21-1f04cb889ebc admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:13.633032 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee42a29-b28c-482a-bf21-1f04cb889ebc admin admin] PATH_INFO: `/v3/users/25c53e3c82724ab4b997635af0ad79e4` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:13.633296 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d3821b-66f0-415d-b1b5-a85496afd95f admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:31:13.634513 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee42a29-b28c-482a-bf21-1f04cb889ebc admin admin] RBAC: Authorizing `identity:delete_user(user_id=25c53e3c82724ab4b997635af0ad79e4)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:31:13.636743 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee42a29-b28c-482a-bf21-1f04cb889ebc admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:31:13.645292 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07ef720-5006-432a-b79a-659576e46ce1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:31:13.645292 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07ef720-5006-432a-b79a-659576e46ce1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:13.645292 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07ef720-5006-432a-b79a-659576e46ce1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:13.645292 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07ef720-5006-432a-b79a-659576e46ce1 admin admin] PATH_INFO: `/v3/users/db5d8789175745d398d0011f616785c9` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:13.645742 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07ef720-5006-432a-b79a-659576e46ce1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=db5d8789175745d398d0011f616785c9)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:31:13.645742 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07ef720-5006-432a-b79a-659576e46ce1 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:31:13.662821 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:31:13.706536 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-6ee42a29-b28c-482a-bf21-1f04cb889ebc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '25c53e3c82724ab4b997635af0ad79e4'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:31:13.716379 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-c07ef720-5006-432a-b79a-659576e46ce1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'db5d8789175745d398d0011f616785c9'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:31:13.719067 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-6ee42a29-b28c-482a-bf21-1f04cb889ebc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '25c53e3c82724ab4b997635af0ad79e4'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:31:13.723182 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-6ee42a29-b28c-482a-bf21-1f04cb889ebc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '25c53e3c82724ab4b997635af0ad79e4'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:31:13.725907 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-c07ef720-5006-432a-b79a-659576e46ce1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'db5d8789175745d398d0011f616785c9'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:31:13.727020 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-c07ef720-5006-432a-b79a-659576e46ce1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'db5d8789175745d398d0011f616785c9'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:31:13.735521 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-6ee42a29-b28c-482a-bf21-1f04cb889ebc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '25c53e3c82724ab4b997635af0ad79e4'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:31:13.736430 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-c07ef720-5006-432a-b79a-659576e46ce1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'db5d8789175745d398d0011f616785c9'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:31:13.737725 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-c07ef720-5006-432a-b79a-659576e46ce1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'db5d8789175745d398d0011f616785c9'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:31:13.773631 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-6ee42a29-b28c-482a-bf21-1f04cb889ebc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '25c53e3c82724ab4b997635af0ad79e4'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:31:13.780785 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1002/1981] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Nov 2 07:31:13 2022] DELETE /identity/v3/users/db5d8789175745d398d0011f616785c9 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 07:31:13.782018 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07ef720-5006-432a-b79a-659576e46ce1 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:31:13.790429 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 980/1982] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Nov 2 07:31:13 2022] DELETE /identity/v3/users/25c53e3c82724ab4b997635af0ad79e4 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:31:13.791846 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e9738c-5308-4bb4-8525-c034e21b294e None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:13.792171 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e9738c-5308-4bb4-8525-c034e21b294e None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:13.792455 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e9738c-5308-4bb4-8525-c034e21b294e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:13.796283 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-c07ef720-5006-432a-b79a-659576e46ce1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:31:13.813940 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-59e9738c-5308-4bb4-8525-c034e21b294e None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:31:13.815344 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-59e9738c-5308-4bb4-8525-c034e21b294e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:31:13.827979 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:31:13.897466 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 981/1983] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:31:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:31:13.897466 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49520d5-2f5a-4919-b606-f4af14d6c779 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:13.897466 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49520d5-2f5a-4919-b606-f4af14d6c779 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:13.897466 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49520d5-2f5a-4919-b606-f4af14d6c779 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:13.905762 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07ef720-5006-432a-b79a-659576e46ce1 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:31:13.917447 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-d49520d5-2f5a-4919-b606-f4af14d6c779 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:31:13.918592 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-d49520d5-2f5a-4919-b606-f4af14d6c779 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:31:13.926943 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 982/1984] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:31:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:31:13.931194 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49520d5-2f5a-4919-b606-f4af14d6c779 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:31:13.971133 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d442b5b5-b0d1-470d-9568-b5f8df04edf3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3ec6ecb85f94ef59762877d9ea33ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:31:13.972063 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d442b5b5-b0d1-470d-9568-b5f8df04edf3 service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:13.972518 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d442b5b5-b0d1-470d-9568-b5f8df04edf3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:13.973044 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d442b5b5-b0d1-470d-9568-b5f8df04edf3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:13.975377 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-d442b5b5-b0d1-470d-9568-b5f8df04edf3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:31:13.989352 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d442b5b5-b0d1-470d-9568-b5f8df04edf3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:31:13.991012 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d442b5b5-b0d1-470d-9568-b5f8df04edf3 service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:31:14.004120 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1003/1985] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:31:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:31:14.005180 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d442b5b5-b0d1-470d-9568-b5f8df04edf3 service neutron] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:31:14.015167 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d442b5b5-b0d1-470d-9568-b5f8df04edf3 service neutron] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:31:14.019067 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27d4eb5-2b1e-4b55-8301-3a5fdecae682 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3ec6ecb85f94ef59762877d9ea33ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:31:14.019757 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27d4eb5-2b1e-4b55-8301-3a5fdecae682 service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:14.020089 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27d4eb5-2b1e-4b55-8301-3a5fdecae682 service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:14.020360 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27d4eb5-2b1e-4b55-8301-3a5fdecae682 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:14.021228 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49520d5-2f5a-4919-b606-f4af14d6c779 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:31:14.026393 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810605e2-bbf6-43a4-bd34-667e097950bd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3ec6ecb85f94ef59762877d9ea33ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:31:14.026955 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810605e2-bbf6-43a4-bd34-667e097950bd service neutron] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:14.027711 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810605e2-bbf6-43a4-bd34-667e097950bd service neutron] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:14.027994 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810605e2-bbf6-43a4-bd34-667e097950bd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:14.035689 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27d4eb5-2b1e-4b55-8301-3a5fdecae682 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:31:14.038062 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27d4eb5-2b1e-4b55-8301-3a5fdecae682 service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:31:14.043739 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810605e2-bbf6-43a4-bd34-667e097950bd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:31:14.046405 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810605e2-bbf6-43a4-bd34-667e097950bd service neutron] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:31:14.055521 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1004/1986] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:31:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:31:14.057203 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 983/1987] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:31:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:31:14.612909 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27d4eb5-2b1e-4b55-8301-3a5fdecae682 service neutron] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:31:14.614453 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-f27d4eb5-2b1e-4b55-8301-3a5fdecae682 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:31:14.628719 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27d4eb5-2b1e-4b55-8301-3a5fdecae682 service neutron] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:31:14.633726 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4c570e-dc9f-4d4d-9210-9a969aaa61ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:31:14.635030 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4c570e-dc9f-4d4d-9210-9a969aaa61ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:14.635468 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4c570e-dc9f-4d4d-9210-9a969aaa61ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:14.635818 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4c570e-dc9f-4d4d-9210-9a969aaa61ac admin admin] PATH_INFO: `/v3/projects/71f746654bee4209b28de275a9ce5116` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:14.637373 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4c570e-dc9f-4d4d-9210-9a969aaa61ac admin admin] RBAC: Authorizing `identity:delete_project(project_id=71f746654bee4209b28de275a9ce5116)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:31:14.640164 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4c570e-dc9f-4d4d-9210-9a969aaa61ac admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:31:14.682149 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:31:14.704260 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-2a4c570e-dc9f-4d4d-9210-9a969aaa61ac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '71f746654bee4209b28de275a9ce5116', 'request_id': 'req-2a4c570e-dc9f-4d4d-9210-9a969aaa61ac'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:31:14.713449 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-2a4c570e-dc9f-4d4d-9210-9a969aaa61ac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '71f746654bee4209b28de275a9ce5116', 'request_id': 'req-2a4c570e-dc9f-4d4d-9210-9a969aaa61ac'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:31:14.741702 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-2a4c570e-dc9f-4d4d-9210-9a969aaa61ac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '71f746654bee4209b28de275a9ce5116', 'request_id': 'req-2a4c570e-dc9f-4d4d-9210-9a969aaa61ac'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:31:14.773753 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-2a4c570e-dc9f-4d4d-9210-9a969aaa61ac admin admin] The token cache is being invalidate because project 71f746654bee4209b28de275a9ce5116 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:31:14.774042 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-2a4c570e-dc9f-4d4d-9210-9a969aaa61ac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:31:14.774796 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-2a4c570e-dc9f-4d4d-9210-9a969aaa61ac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:31:14.777741 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1005/1988] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Nov 2 07:31:14 2022] DELETE /identity/v3/projects/71f746654bee4209b28de275a9ce5116 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:31:14.839545 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810605e2-bbf6-43a4-bd34-667e097950bd service neutron] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:31:14.871320 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-810605e2-bbf6-43a4-bd34-667e097950bd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:31:14.913411 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810605e2-bbf6-43a4-bd34-667e097950bd service neutron] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:31:15.034011 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e0c168-925b-4e91-8c80-41482ce5b5f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:31:15.034737 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e0c168-925b-4e91-8c80-41482ce5b5f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:15.035045 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e0c168-925b-4e91-8c80-41482ce5b5f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:15.035326 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e0c168-925b-4e91-8c80-41482ce5b5f9 admin admin] PATH_INFO: `/v3/projects/3b6b21916b764c9fb3594c366e097f2e` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:15.036437 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e0c168-925b-4e91-8c80-41482ce5b5f9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3b6b21916b764c9fb3594c366e097f2e)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:31:15.038390 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e0c168-925b-4e91-8c80-41482ce5b5f9 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:31:15.095684 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-75e0c168-925b-4e91-8c80-41482ce5b5f9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3b6b21916b764c9fb3594c366e097f2e', 'request_id': 'req-75e0c168-925b-4e91-8c80-41482ce5b5f9'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:31:15.103178 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-75e0c168-925b-4e91-8c80-41482ce5b5f9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3b6b21916b764c9fb3594c366e097f2e', 'request_id': 'req-75e0c168-925b-4e91-8c80-41482ce5b5f9'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:31:15.105267 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:31:15.113404 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-75e0c168-925b-4e91-8c80-41482ce5b5f9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3b6b21916b764c9fb3594c366e097f2e', 'request_id': 'req-75e0c168-925b-4e91-8c80-41482ce5b5f9'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:31:15.128429 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-75e0c168-925b-4e91-8c80-41482ce5b5f9 admin admin] The token cache is being invalidate because project 3b6b21916b764c9fb3594c366e097f2e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81501) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:31:15.128739 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-75e0c168-925b-4e91-8c80-41482ce5b5f9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:31:15.129437 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-75e0c168-925b-4e91-8c80-41482ce5b5f9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:31:15.131396 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 984/1989] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Nov 2 07:31:14 2022] DELETE /identity/v3/projects/3b6b21916b764c9fb3594c366e097f2e => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:31:15.135076 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4c570e-dc9f-4d4d-9210-9a969aaa61ac admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:31:15.136498 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-2a4c570e-dc9f-4d4d-9210-9a969aaa61ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:31:15.149866 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4c570e-dc9f-4d4d-9210-9a969aaa61ac admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:31:15.216115 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e6e086-5fd8-488c-9833-5be9956befd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:31:15.217721 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e6e086-5fd8-488c-9833-5be9956befd2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:15.217980 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e6e086-5fd8-488c-9833-5be9956befd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:15.218256 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e6e086-5fd8-488c-9833-5be9956befd2 admin admin] PATH_INFO: `/v3/users/4cdb247032804c92a7c16074ffa2b6de` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:15.219442 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e6e086-5fd8-488c-9833-5be9956befd2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4cdb247032804c92a7c16074ffa2b6de)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:31:15.221484 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e6e086-5fd8-488c-9833-5be9956befd2 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:31:15.275394 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-25e6e086-5fd8-488c-9833-5be9956befd2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4cdb247032804c92a7c16074ffa2b6de'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:31:15.288158 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-25e6e086-5fd8-488c-9833-5be9956befd2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4cdb247032804c92a7c16074ffa2b6de'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:31:15.289232 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-25e6e086-5fd8-488c-9833-5be9956befd2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4cdb247032804c92a7c16074ffa2b6de'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:31:15.298628 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-25e6e086-5fd8-488c-9833-5be9956befd2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4cdb247032804c92a7c16074ffa2b6de'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:31:15.299342 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-25e6e086-5fd8-488c-9833-5be9956befd2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4cdb247032804c92a7c16074ffa2b6de'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:31:15.320379 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1006/1990] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Nov 2 07:31:15 2022] DELETE /identity/v3/users/4cdb247032804c92a7c16074ffa2b6de => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:31:15.639332 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e0c168-925b-4e91-8c80-41482ce5b5f9 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:31:15.647173 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-75e0c168-925b-4e91-8c80-41482ce5b5f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:31:15.674094 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e0c168-925b-4e91-8c80-41482ce5b5f9 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:31:15.739336 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eeb11ea-f120-4b26-8971-b4aef026a8ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:31:15.740071 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eeb11ea-f120-4b26-8971-b4aef026a8ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:15.740429 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eeb11ea-f120-4b26-8971-b4aef026a8ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:15.740788 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eeb11ea-f120-4b26-8971-b4aef026a8ff admin admin] PATH_INFO: `/v3/projects/20a877fe29504f23b25072daff032068` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:15.743164 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eeb11ea-f120-4b26-8971-b4aef026a8ff admin admin] RBAC: Authorizing `identity:delete_project(project_id=20a877fe29504f23b25072daff032068)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:31:15.744117 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eeb11ea-f120-4b26-8971-b4aef026a8ff admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:31:15.815395 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-2eeb11ea-f120-4b26-8971-b4aef026a8ff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '20a877fe29504f23b25072daff032068', 'request_id': 'req-2eeb11ea-f120-4b26-8971-b4aef026a8ff'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:31:15.820465 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f593559-f088-4d4f-b57f-51294abbc252 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:15.820981 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f593559-f088-4d4f-b57f-51294abbc252 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:15.821078 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f593559-f088-4d4f-b57f-51294abbc252 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:15.830811 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-2eeb11ea-f120-4b26-8971-b4aef026a8ff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '20a877fe29504f23b25072daff032068', 'request_id': 'req-2eeb11ea-f120-4b26-8971-b4aef026a8ff'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:31:15.839966 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-2eeb11ea-f120-4b26-8971-b4aef026a8ff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '20a877fe29504f23b25072daff032068', 'request_id': 'req-2eeb11ea-f120-4b26-8971-b4aef026a8ff'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:31:15.856388 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-2eeb11ea-f120-4b26-8971-b4aef026a8ff admin admin] The token cache is being invalidate because project 20a877fe29504f23b25072daff032068 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81501) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:31:15.856826 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-2eeb11ea-f120-4b26-8971-b4aef026a8ff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:31:15.857746 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-2eeb11ea-f120-4b26-8971-b4aef026a8ff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:31:15.860403 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 985/1991] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Nov 2 07:31:15 2022] DELETE /identity/v3/projects/20a877fe29504f23b25072daff032068 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:31:15.864490 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eeb11ea-f120-4b26-8971-b4aef026a8ff admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:31:15.866198 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-2eeb11ea-f120-4b26-8971-b4aef026a8ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:31:15.877626 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-9f593559-f088-4d4f-b57f-51294abbc252 None None] MFA Rules not processed for user `0b55941561934c818a2b5eb2c9a19d22`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:31:15.880582 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-9f593559-f088-4d4f-b57f-51294abbc252 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:31:15.922226 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:31:15.995761 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eeb11ea-f120-4b26-8971-b4aef026a8ff admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:31:16.001729 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1a6dab-1ceb-4eb0-97a6-363b8c28e4f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:31:16.002393 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1a6dab-1ceb-4eb0-97a6-363b8c28e4f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:16.002754 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1a6dab-1ceb-4eb0-97a6-363b8c28e4f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:16.003010 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1a6dab-1ceb-4eb0-97a6-363b8c28e4f4 admin admin] PATH_INFO: `/v3/users/80e7bef52ecb483ca9f00af3b558c29e` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:16.004096 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1a6dab-1ceb-4eb0-97a6-363b8c28e4f4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=80e7bef52ecb483ca9f00af3b558c29e)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:31:16.006015 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1a6dab-1ceb-4eb0-97a6-363b8c28e4f4 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:31:16.015415 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1007/1992] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:31:15 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:31:16.020411 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:INSERT=2,keystone:DELETE=11,keystone:SELECT=87 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:31:16.026486 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f593559-f088-4d4f-b57f-51294abbc252 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:31:16.027837 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-9f593559-f088-4d4f-b57f-51294abbc252 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:31:16.058305 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-ce1a6dab-1ceb-4eb0-97a6-363b8c28e4f4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '80e7bef52ecb483ca9f00af3b558c29e'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:31:16.066663 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-ce1a6dab-1ceb-4eb0-97a6-363b8c28e4f4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '80e7bef52ecb483ca9f00af3b558c29e'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:31:16.067295 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-ce1a6dab-1ceb-4eb0-97a6-363b8c28e4f4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '80e7bef52ecb483ca9f00af3b558c29e'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:31:16.074091 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-ce1a6dab-1ceb-4eb0-97a6-363b8c28e4f4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '80e7bef52ecb483ca9f00af3b558c29e'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:31:16.087360 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-ce1a6dab-1ceb-4eb0-97a6-363b8c28e4f4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '80e7bef52ecb483ca9f00af3b558c29e'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:31:16.088307 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f593559-f088-4d4f-b57f-51294abbc252 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:31:16.092628 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104fa59a-7a43-4223-887a-19b94f8d7467 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43bb326fb82e4d2c83fb2eaf91860dc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:31:16.093232 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104fa59a-7a43-4223-887a-19b94f8d7467 service cinder] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:16.093516 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104fa59a-7a43-4223-887a-19b94f8d7467 service cinder] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:16.093752 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 986/1993] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Nov 2 07:31:15 2022] DELETE /identity/v3/users/80e7bef52ecb483ca9f00af3b558c29e => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 07:31:16.093848 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104fa59a-7a43-4223-887a-19b94f8d7467 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:16.096247 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-104fa59a-7a43-4223-887a-19b94f8d7467 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:31:16.108965 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104fa59a-7a43-4223-887a-19b94f8d7467 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:31:16.110502 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104fa59a-7a43-4223-887a-19b94f8d7467 service cinder] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:31:16.120008 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1008/1994] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:31:16 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 07:31:16.418786 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1a6dab-1ceb-4eb0-97a6-363b8c28e4f4 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:31:16.420708 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-ce1a6dab-1ceb-4eb0-97a6-363b8c28e4f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:31:16.549735 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1a6dab-1ceb-4eb0-97a6-363b8c28e4f4 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:31:16.554652 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb0471a-62f2-4cef-811a-a3da697cd343 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:31:16.555452 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb0471a-62f2-4cef-811a-a3da697cd343 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:16.555806 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb0471a-62f2-4cef-811a-a3da697cd343 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:16.556149 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb0471a-62f2-4cef-811a-a3da697cd343 admin admin] PATH_INFO: `/v3/projects/2035a822d70a4ef685b052738d638027` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:16.557434 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb0471a-62f2-4cef-811a-a3da697cd343 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2035a822d70a4ef685b052738d638027)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:31:16.559666 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb0471a-62f2-4cef-811a-a3da697cd343 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:31:16.610789 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-cfb0471a-62f2-4cef-811a-a3da697cd343 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2035a822d70a4ef685b052738d638027', 'request_id': 'req-cfb0471a-62f2-4cef-811a-a3da697cd343'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:31:16.618096 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-cfb0471a-62f2-4cef-811a-a3da697cd343 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2035a822d70a4ef685b052738d638027', 'request_id': 'req-cfb0471a-62f2-4cef-811a-a3da697cd343'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:31:16.627633 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-cfb0471a-62f2-4cef-811a-a3da697cd343 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2035a822d70a4ef685b052738d638027', 'request_id': 'req-cfb0471a-62f2-4cef-811a-a3da697cd343'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:31:16.645503 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-cfb0471a-62f2-4cef-811a-a3da697cd343 admin admin] The token cache is being invalidate because project 2035a822d70a4ef685b052738d638027 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81501) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:31:16.658512 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-cfb0471a-62f2-4cef-811a-a3da697cd343 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:31:16.659783 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-cfb0471a-62f2-4cef-811a-a3da697cd343 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:31:16.661635 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 987/1995] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Nov 2 07:31:16 2022] DELETE /identity/v3/projects/2035a822d70a4ef685b052738d638027 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:31:16.668228 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59916cbc-6819-4968-a38d-181726b34073 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:16.670448 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59916cbc-6819-4968-a38d-181726b34073 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:16.670448 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59916cbc-6819-4968-a38d-181726b34073 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:16.711712 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-59916cbc-6819-4968-a38d-181726b34073 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:31:16.713661 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-59916cbc-6819-4968-a38d-181726b34073 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:31:16.777864 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1009/1996] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:31:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:31:16.785884 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ced835-e449-4860-beca-f783b644eec8 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:16.786174 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ced835-e449-4860-beca-f783b644eec8 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:16.786484 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ced835-e449-4860-beca-f783b644eec8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:16.807628 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-08ced835-e449-4860-beca-f783b644eec8 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:31:16.808837 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-08ced835-e449-4860-beca-f783b644eec8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:31:16.817529 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 988/1997] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:31:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:31:16.821782 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59916cbc-6819-4968-a38d-181726b34073 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:31:16.837939 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59916cbc-6819-4968-a38d-181726b34073 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:31:16.842717 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804ef16b-6b44-43e9-941d-56f2c69b897d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:31:16.843336 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804ef16b-6b44-43e9-941d-56f2c69b897d admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:16.843618 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804ef16b-6b44-43e9-941d-56f2c69b897d admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:16.843891 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804ef16b-6b44-43e9-941d-56f2c69b897d admin admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:16.844541 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804ef16b-6b44-43e9-941d-56f2c69b897d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:31:16.844842 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804ef16b-6b44-43e9-941d-56f2c69b897d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:31:16.846460 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804ef16b-6b44-43e9-941d-56f2c69b897d admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:31:16.859181 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1010/1998] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Nov 2 07:31:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:31:16.863399 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ced835-e449-4860-beca-f783b644eec8 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:31:16.874493 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ced835-e449-4860-beca-f783b644eec8 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:31:16.879575 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88107ab-5c60-49c1-b736-dee2e615b996 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:31:16.880358 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88107ab-5c60-49c1-b736-dee2e615b996 admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:16.880633 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88107ab-5c60-49c1-b736-dee2e615b996 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:16.880901 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88107ab-5c60-49c1-b736-dee2e615b996 admin admin] PATH_INFO: `/v3/projects` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:16.881548 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88107ab-5c60-49c1-b736-dee2e615b996 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:31:16.883558 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88107ab-5c60-49c1-b736-dee2e615b996 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:31:16.898936 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:31:16.919200 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 989/1999] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Nov 2 07:31:16 2022] POST /identity/v3/projects => generated 379 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:31:16.922537 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804ef16b-6b44-43e9-941d-56f2c69b897d admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:31:16.932924 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804ef16b-6b44-43e9-941d-56f2c69b897d admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:31:16.982760 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:31:16.989801 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845ce40a-68f1-42bb-84c1-fd90b4f61d96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:31:16.990434 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845ce40a-68f1-42bb-84c1-fd90b4f61d96 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:16.990712 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845ce40a-68f1-42bb-84c1-fd90b4f61d96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:16.990985 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845ce40a-68f1-42bb-84c1-fd90b4f61d96 admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:16.991609 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845ce40a-68f1-42bb-84c1-fd90b4f61d96 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:31:16.993718 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845ce40a-68f1-42bb-84c1-fd90b4f61d96 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:31:17.015553 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1011/2000] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Nov 2 07:31:16 2022] POST /identity/v3/users => generated 290 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 07:31:17.019851 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88107ab-5c60-49c1-b736-dee2e615b996 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:31:17.030878 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88107ab-5c60-49c1-b736-dee2e615b996 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:31:17.035836 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a33eb4-0f16-4709-b316-3f318969c095 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:31:17.036495 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a33eb4-0f16-4709-b316-3f318969c095 admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:17.036778 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a33eb4-0f16-4709-b316-3f318969c095 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:17.037052 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a33eb4-0f16-4709-b316-3f318969c095 admin admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:17.037685 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a33eb4-0f16-4709-b316-3f318969c095 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:31:17.039310 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a33eb4-0f16-4709-b316-3f318969c095 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:31:17.051043 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 990/2001] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:31:17 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:31:17.055063 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845ce40a-68f1-42bb-84c1-fd90b4f61d96 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:31:17.065561 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845ce40a-68f1-42bb-84c1-fd90b4f61d96 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:31:17.070222 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d286908-f1ec-48a3-a9cb-67a4f556ed08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:31:17.070868 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d286908-f1ec-48a3-a9cb-67a4f556ed08 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:17.071150 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d286908-f1ec-48a3-a9cb-67a4f556ed08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:17.071424 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d286908-f1ec-48a3-a9cb-67a4f556ed08 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:17.072060 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d286908-f1ec-48a3-a9cb-67a4f556ed08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:31:17.073645 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d286908-f1ec-48a3-a9cb-67a4f556ed08 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:31:17.085240 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1012/2002] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:31:17 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:31:17.089605 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a33eb4-0f16-4709-b316-3f318969c095 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:31:17.099690 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a33eb4-0f16-4709-b316-3f318969c095 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:31:17.104245 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4c3297-3151-4f8a-9838-93764c2dffc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:31:17.104898 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4c3297-3151-4f8a-9838-93764c2dffc8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:17.105177 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4c3297-3151-4f8a-9838-93764c2dffc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:17.105444 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4c3297-3151-4f8a-9838-93764c2dffc8 admin admin] PATH_INFO: `/v3/projects/0f2be348b8f44fefbc65fb902427463f/users/a8ded0b2c2764c2181c7113620a7a711/roles/b494fedd00f24f0b84a529e8672a156a` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:17.121538 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4c3297-3151-4f8a-9838-93764c2dffc8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f2be348b8f44fefbc65fb902427463f, user_id=a8ded0b2c2764c2181c7113620a7a711, role_id=b494fedd00f24f0b84a529e8672a156a)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:31:17.124602 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4c3297-3151-4f8a-9838-93764c2dffc8 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:31:17.138662 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 991/2003] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Nov 2 07:31:17 2022] PUT /identity/v3/projects/0f2be348b8f44fefbc65fb902427463f/users/a8ded0b2c2764c2181c7113620a7a711/roles/b494fedd00f24f0b84a529e8672a156a => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:31:17.149577 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95825df0-a524-49a4-a6a0-024a292ef01a None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:17.149892 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95825df0-a524-49a4-a6a0-024a292ef01a None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:17.150133 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95825df0-a524-49a4-a6a0-024a292ef01a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:17.179034 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-95825df0-a524-49a4-a6a0-024a292ef01a None None] MFA Rules not processed for user `a8ded0b2c2764c2181c7113620a7a711`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:31:17.180246 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-95825df0-a524-49a4-a6a0-024a292ef01a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:31:17.243465 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1013/2004] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:31:17 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:31:17.248738 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4c3297-3151-4f8a-9838-93764c2dffc8 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:31:17.308508 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4c3297-3151-4f8a-9838-93764c2dffc8 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:31:17.313407 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092e1cae-858d-4385-9325-d2fb524de74b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:31:17.314007 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092e1cae-858d-4385-9325-d2fb524de74b admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:17.314279 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092e1cae-858d-4385-9325-d2fb524de74b admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:17.314565 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092e1cae-858d-4385-9325-d2fb524de74b admin admin] PATH_INFO: `/v3/projects` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:17.315192 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092e1cae-858d-4385-9325-d2fb524de74b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:31:17.317077 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092e1cae-858d-4385-9325-d2fb524de74b admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:31:17.344020 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 992/2005] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Nov 2 07:31:17 2022] POST /identity/v3/projects => generated 379 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:31:17.348056 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95825df0-a524-49a4-a6a0-024a292ef01a None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:31:17.359353 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95825df0-a524-49a4-a6a0-024a292ef01a None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:31:17.415319 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1490c9-7eed-4ddc-a99d-ded6e8f5856e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:31:17.415875 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1490c9-7eed-4ddc-a99d-ded6e8f5856e admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:17.416153 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1490c9-7eed-4ddc-a99d-ded6e8f5856e admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:17.416453 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1490c9-7eed-4ddc-a99d-ded6e8f5856e admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:17.417081 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1490c9-7eed-4ddc-a99d-ded6e8f5856e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:31:17.419257 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1490c9-7eed-4ddc-a99d-ded6e8f5856e admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:31:17.442128 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1014/2006] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Nov 2 07:31:17 2022] POST /identity/v3/users => generated 290 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:31:17.446127 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092e1cae-858d-4385-9325-d2fb524de74b admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:31:17.457498 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092e1cae-858d-4385-9325-d2fb524de74b admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:31:17.462379 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e16969-b273-4551-9376-f32dab32e713 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:31:17.463006 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e16969-b273-4551-9376-f32dab32e713 admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:17.463288 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e16969-b273-4551-9376-f32dab32e713 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:17.463559 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e16969-b273-4551-9376-f32dab32e713 admin admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:17.464201 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e16969-b273-4551-9376-f32dab32e713 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:31:17.465811 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e16969-b273-4551-9376-f32dab32e713 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:31:17.477783 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 993/2007] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:31:17 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 07:31:17.481914 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1490c9-7eed-4ddc-a99d-ded6e8f5856e admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:31:17.492446 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1490c9-7eed-4ddc-a99d-ded6e8f5856e admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:31:17.497391 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178e8cb2-e321-4bba-8767-eacc6a75436c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:31:17.498124 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178e8cb2-e321-4bba-8767-eacc6a75436c admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:17.498686 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178e8cb2-e321-4bba-8767-eacc6a75436c admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:17.498686 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178e8cb2-e321-4bba-8767-eacc6a75436c admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:17.499355 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178e8cb2-e321-4bba-8767-eacc6a75436c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:31:17.500924 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178e8cb2-e321-4bba-8767-eacc6a75436c admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:31:17.512060 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1015/2008] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:31:17 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:31:17.516290 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e16969-b273-4551-9376-f32dab32e713 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:31:17.526652 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e16969-b273-4551-9376-f32dab32e713 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:31:17.532233 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1ea676-13b1-4596-9aa5-77dab93a73ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:31:17.532894 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1ea676-13b1-4596-9aa5-77dab93a73ad admin admin] REQUEST_METHOD: `PUT` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:17.533188 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1ea676-13b1-4596-9aa5-77dab93a73ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:17.533586 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1ea676-13b1-4596-9aa5-77dab93a73ad admin admin] PATH_INFO: `/v3/projects/1b4fccca12b44b0ab55dc1f67ee85305/users/9099f5b195d54445b92ad14796bc7887/roles/b494fedd00f24f0b84a529e8672a156a` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:17.554272 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1ea676-13b1-4596-9aa5-77dab93a73ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=1b4fccca12b44b0ab55dc1f67ee85305, user_id=9099f5b195d54445b92ad14796bc7887, role_id=b494fedd00f24f0b84a529e8672a156a)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:31:17.557109 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1ea676-13b1-4596-9aa5-77dab93a73ad admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:31:17.561322 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9809c56-992f-44ff-a080-b83c10df5c1d None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:17.561573 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9809c56-992f-44ff-a080-b83c10df5c1d None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:17.561850 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9809c56-992f-44ff-a080-b83c10df5c1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:17.571774 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 994/2009] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Nov 2 07:31:17 2022] PUT /identity/v3/projects/1b4fccca12b44b0ab55dc1f67ee85305/users/9099f5b195d54445b92ad14796bc7887/roles/b494fedd00f24f0b84a529e8672a156a => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:31:17.583492 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-f9809c56-992f-44ff-a080-b83c10df5c1d None None] MFA Rules not processed for user `0b55941561934c818a2b5eb2c9a19d22`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:31:17.583973 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47f40f9-65e4-4994-9871-562a52417d69 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:17.584259 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47f40f9-65e4-4994-9871-562a52417d69 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:17.584533 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47f40f9-65e4-4994-9871-562a52417d69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:17.584726 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-f9809c56-992f-44ff-a080-b83c10df5c1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:31:17.619613 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-d47f40f9-65e4-4994-9871-562a52417d69 None None] MFA Rules not processed for user `9099f5b195d54445b92ad14796bc7887`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:31:17.620888 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-d47f40f9-65e4-4994-9871-562a52417d69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:31:17.636066 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:31:17.661756 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1016/2010] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:31:17 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:31:17.705402 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:31:17.715395 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 995/2011] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:31:17 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:31:17.717288 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9809c56-992f-44ff-a080-b83c10df5c1d None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:31:17.786755 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9809c56-992f-44ff-a080-b83c10df5c1d None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:31:17.791429 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ab946e-a794-4643-ad1b-a152ea44a336 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:31:17.792039 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ab946e-a794-4643-ad1b-a152ea44a336 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:17.792695 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ab946e-a794-4643-ad1b-a152ea44a336 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:17.792695 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ab946e-a794-4643-ad1b-a152ea44a336 admin admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:17.793214 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ab946e-a794-4643-ad1b-a152ea44a336 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:31:17.795265 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ab946e-a794-4643-ad1b-a152ea44a336 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:31:17.827748 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1017/2012] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Nov 2 07:31:17 2022] POST /identity/v3/projects => generated 377 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:31:17.829184 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47f40f9-65e4-4994-9871-562a52417d69 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:31:17.840465 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47f40f9-65e4-4994-9871-562a52417d69 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:31:17.925031 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026b1e1b-1e12-4c2d-9413-ea797cf8e0e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:31:17.926006 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026b1e1b-1e12-4c2d-9413-ea797cf8e0e8 admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:17.926288 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026b1e1b-1e12-4c2d-9413-ea797cf8e0e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:17.926598 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026b1e1b-1e12-4c2d-9413-ea797cf8e0e8 admin admin] PATH_INFO: `/v3/users` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:17.927236 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026b1e1b-1e12-4c2d-9413-ea797cf8e0e8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:31:17.929177 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026b1e1b-1e12-4c2d-9413-ea797cf8e0e8 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:31:17.955902 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 996/2013] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Nov 2 07:31:17 2022] POST /identity/v3/users => generated 295 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:31:17.957939 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ab946e-a794-4643-ad1b-a152ea44a336 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:31:17.969151 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ab946e-a794-4643-ad1b-a152ea44a336 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:31:17.974051 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708a2387-e4d5-4fb2-9c49-6131a2658cc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:31:17.974839 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708a2387-e4d5-4fb2-9c49-6131a2658cc3 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:17.975184 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708a2387-e4d5-4fb2-9c49-6131a2658cc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:17.975527 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708a2387-e4d5-4fb2-9c49-6131a2658cc3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:17.976239 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708a2387-e4d5-4fb2-9c49-6131a2658cc3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:31:17.977878 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708a2387-e4d5-4fb2-9c49-6131a2658cc3 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:31:17.989514 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1018/2014] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:31:17 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:31:17.994041 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026b1e1b-1e12-4c2d-9413-ea797cf8e0e8 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:31:18.005457 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026b1e1b-1e12-4c2d-9413-ea797cf8e0e8 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:31:18.010174 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9f9f5e-7fce-4abd-b128-1ee489afadb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:31:18.010879 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9f9f5e-7fce-4abd-b128-1ee489afadb1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:18.011155 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9f9f5e-7fce-4abd-b128-1ee489afadb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:18.011430 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9f9f5e-7fce-4abd-b128-1ee489afadb1 admin admin] PATH_INFO: `/v3/projects/aaef001e3398403c9dc419142795859b/users/93e457f801114e4db1989a9d6ac3f50d/roles/c4bf019fc6c44c71a6c984f6d0bab75f` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:18.028412 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9f9f5e-7fce-4abd-b128-1ee489afadb1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aaef001e3398403c9dc419142795859b, user_id=93e457f801114e4db1989a9d6ac3f50d, role_id=c4bf019fc6c44c71a6c984f6d0bab75f)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:31:18.036700 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9f9f5e-7fce-4abd-b128-1ee489afadb1 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:31:18.057576 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 997/2015] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Nov 2 07:31:17 2022] PUT /identity/v3/projects/aaef001e3398403c9dc419142795859b/users/93e457f801114e4db1989a9d6ac3f50d/roles/c4bf019fc6c44c71a6c984f6d0bab75f => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 07:31:18.062110 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708a2387-e4d5-4fb2-9c49-6131a2658cc3 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:31:18.074399 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708a2387-e4d5-4fb2-9c49-6131a2658cc3 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:31:18.136906 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a576b5-931e-4bf5-8c26-4f2df83bef1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:31:18.137659 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a576b5-931e-4bf5-8c26-4f2df83bef1c admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:18.137942 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a576b5-931e-4bf5-8c26-4f2df83bef1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:18.138932 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a576b5-931e-4bf5-8c26-4f2df83bef1c admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:18.139665 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a576b5-931e-4bf5-8c26-4f2df83bef1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:31:18.141259 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a576b5-931e-4bf5-8c26-4f2df83bef1c admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:31:18.159342 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1019/2016] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:31:18 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:31:18.160823 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9f9f5e-7fce-4abd-b128-1ee489afadb1 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:31:18.179269 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9f9f5e-7fce-4abd-b128-1ee489afadb1 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:31:18.183440 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970b7e51-90d7-4502-abd9-d6fb14033172 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:31:18.184130 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970b7e51-90d7-4502-abd9-d6fb14033172 admin admin] REQUEST_METHOD: `PUT` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:18.184407 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970b7e51-90d7-4502-abd9-d6fb14033172 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:18.184675 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970b7e51-90d7-4502-abd9-d6fb14033172 admin admin] PATH_INFO: `/v3/projects/aaef001e3398403c9dc419142795859b/users/93e457f801114e4db1989a9d6ac3f50d/roles/c4bf019fc6c44c71a6c984f6d0bab75f` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:18.186808 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970b7e51-90d7-4502-abd9-d6fb14033172 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aaef001e3398403c9dc419142795859b, user_id=93e457f801114e4db1989a9d6ac3f50d, role_id=c4bf019fc6c44c71a6c984f6d0bab75f)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:31:18.189443 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970b7e51-90d7-4502-abd9-d6fb14033172 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:31:18.200914 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 998/2017] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Nov 2 07:31:18 2022] PUT /identity/v3/projects/aaef001e3398403c9dc419142795859b/users/93e457f801114e4db1989a9d6ac3f50d/roles/c4bf019fc6c44c71a6c984f6d0bab75f => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:31:18.215278 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcbf048-cf9d-4a3f-bac1-bc6bd18a4d5e None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:18.215694 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcbf048-cf9d-4a3f-bac1-bc6bd18a4d5e None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:18.216037 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcbf048-cf9d-4a3f-bac1-bc6bd18a4d5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:18.248807 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-0fcbf048-cf9d-4a3f-bac1-bc6bd18a4d5e None None] MFA Rules not processed for user `93e457f801114e4db1989a9d6ac3f50d`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:31:18.250157 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-0fcbf048-cf9d-4a3f-bac1-bc6bd18a4d5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:31:18.298440 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:31:18.324318 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1020/2018] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:31:18 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:31:18.336028 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970b7e51-90d7-4502-abd9-d6fb14033172 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:31:18.337743 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-970b7e51-90d7-4502-abd9-d6fb14033172 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:31:18.398687 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970b7e51-90d7-4502-abd9-d6fb14033172 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:31:18.402448 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d74bd12-d06a-4057-bcc1-84f1a76e494b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43bb326fb82e4d2c83fb2eaf91860dc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:31:18.403067 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d74bd12-d06a-4057-bcc1-84f1a76e494b service cinder] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:18.403345 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d74bd12-d06a-4057-bcc1-84f1a76e494b service cinder] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:18.403616 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d74bd12-d06a-4057-bcc1-84f1a76e494b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:18.415270 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d74bd12-d06a-4057-bcc1-84f1a76e494b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:31:18.416704 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d74bd12-d06a-4057-bcc1-84f1a76e494b service cinder] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:31:18.427192 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 999/2019] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:31:18 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:31:18.471055 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcbf048-cf9d-4a3f-bac1-bc6bd18a4d5e None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:31:18.472569 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-0fcbf048-cf9d-4a3f-bac1-bc6bd18a4d5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:31:18.484440 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcbf048-cf9d-4a3f-bac1-bc6bd18a4d5e None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:31:18.487991 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0be4f9-19cc-4db7-a9ae-1836fe2a1eda service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43bb326fb82e4d2c83fb2eaf91860dc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:31:18.488587 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0be4f9-19cc-4db7-a9ae-1836fe2a1eda service cinder] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:18.488863 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0be4f9-19cc-4db7-a9ae-1836fe2a1eda service cinder] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:18.489147 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0be4f9-19cc-4db7-a9ae-1836fe2a1eda service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:18.553676 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0be4f9-19cc-4db7-a9ae-1836fe2a1eda service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:31:18.555143 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0be4f9-19cc-4db7-a9ae-1836fe2a1eda service cinder] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:31:18.564604 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1021/2020] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:31:18 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:31:21.760407 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d74bd12-d06a-4057-bcc1-84f1a76e494b service cinder] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:31:21.831261 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:31:21.843359 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d74bd12-d06a-4057-bcc1-84f1a76e494b service cinder] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:31:21.848325 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b3d423-e6c2-49db-a90c-e058a6af880b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:31:21.848992 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b3d423-e6c2-49db-a90c-e058a6af880b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:21.849271 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b3d423-e6c2-49db-a90c-e058a6af880b admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:21.849542 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b3d423-e6c2-49db-a90c-e058a6af880b admin admin] PATH_INFO: `/v3/users/a8ded0b2c2764c2181c7113620a7a711` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:21.850702 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b3d423-e6c2-49db-a90c-e058a6af880b admin admin] RBAC: Authorizing `identity:delete_user(user_id=a8ded0b2c2764c2181c7113620a7a711)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:31:21.852607 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b3d423-e6c2-49db-a90c-e058a6af880b admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:31:21.902732 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-a5b3d423-e6c2-49db-a90c-e058a6af880b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a8ded0b2c2764c2181c7113620a7a711'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:31:21.911260 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-a5b3d423-e6c2-49db-a90c-e058a6af880b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a8ded0b2c2764c2181c7113620a7a711'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:31:21.911926 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-a5b3d423-e6c2-49db-a90c-e058a6af880b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a8ded0b2c2764c2181c7113620a7a711'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:31:21.919710 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-a5b3d423-e6c2-49db-a90c-e058a6af880b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a8ded0b2c2764c2181c7113620a7a711'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:31:21.932094 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-a5b3d423-e6c2-49db-a90c-e058a6af880b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a8ded0b2c2764c2181c7113620a7a711'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:31:21.940475 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1000/2021] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Nov 2 07:31:21 2022] DELETE /identity/v3/users/a8ded0b2c2764c2181c7113620a7a711 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:31:21.945113 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431c416a-09f5-4978-883f-ba5cfc6a03cf None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:21.945525 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431c416a-09f5-4978-883f-ba5cfc6a03cf None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:21.945806 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431c416a-09f5-4978-883f-ba5cfc6a03cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:21.966854 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-431c416a-09f5-4978-883f-ba5cfc6a03cf None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:31:21.968046 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-431c416a-09f5-4978-883f-ba5cfc6a03cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:31:22.044302 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1022/2022] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:31:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:31:22.055679 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b3d423-e6c2-49db-a90c-e058a6af880b admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:31:22.057185 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-a5b3d423-e6c2-49db-a90c-e058a6af880b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:31:22.118783 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b3d423-e6c2-49db-a90c-e058a6af880b admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:31:22.123465 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6438885d-70a6-4ab6-8b07-6e0b18220f0e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3ec6ecb85f94ef59762877d9ea33ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:31:22.124088 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6438885d-70a6-4ab6-8b07-6e0b18220f0e service neutron] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:22.124368 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6438885d-70a6-4ab6-8b07-6e0b18220f0e service neutron] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:22.124637 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6438885d-70a6-4ab6-8b07-6e0b18220f0e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:22.139651 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6438885d-70a6-4ab6-8b07-6e0b18220f0e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:31:22.141200 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6438885d-70a6-4ab6-8b07-6e0b18220f0e service neutron] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:31:22.152643 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1001/2023] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:31:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:31:22.612320 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431c416a-09f5-4978-883f-ba5cfc6a03cf None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:31:22.613960 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-431c416a-09f5-4978-883f-ba5cfc6a03cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:31:22.634130 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431c416a-09f5-4978-883f-ba5cfc6a03cf None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:31:22.643552 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1f112b-9546-4627-863e-07af4ad85f8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:31:22.644647 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1f112b-9546-4627-863e-07af4ad85f8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:22.644976 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1f112b-9546-4627-863e-07af4ad85f8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:22.645452 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1f112b-9546-4627-863e-07af4ad85f8b admin admin] PATH_INFO: `/v3/projects/0f2be348b8f44fefbc65fb902427463f` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:22.647114 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1f112b-9546-4627-863e-07af4ad85f8b admin admin] RBAC: Authorizing `identity:delete_project(project_id=0f2be348b8f44fefbc65fb902427463f)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:31:22.649417 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1f112b-9546-4627-863e-07af4ad85f8b admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:31:22.690585 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb74dc8-b3f8-471a-8716-036aeb13bf54 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:22.691401 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb74dc8-b3f8-471a-8716-036aeb13bf54 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:22.691635 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb74dc8-b3f8-471a-8716-036aeb13bf54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:22.723727 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-8bb74dc8-b3f8-471a-8716-036aeb13bf54 None None] MFA Rules not processed for user `ff4736f3d50746b0bf5d6a917b2ca0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:31:22.725037 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-8bb74dc8-b3f8-471a-8716-036aeb13bf54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:31:22.731588 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-da1f112b-9546-4627-863e-07af4ad85f8b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0f2be348b8f44fefbc65fb902427463f', 'request_id': 'req-da1f112b-9546-4627-863e-07af4ad85f8b'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:31:22.741524 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-da1f112b-9546-4627-863e-07af4ad85f8b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0f2be348b8f44fefbc65fb902427463f', 'request_id': 'req-da1f112b-9546-4627-863e-07af4ad85f8b'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:31:22.752727 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-da1f112b-9546-4627-863e-07af4ad85f8b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0f2be348b8f44fefbc65fb902427463f', 'request_id': 'req-da1f112b-9546-4627-863e-07af4ad85f8b'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:31:22.774423 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-da1f112b-9546-4627-863e-07af4ad85f8b admin admin] The token cache is being invalidate because project 0f2be348b8f44fefbc65fb902427463f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:31:22.774802 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-da1f112b-9546-4627-863e-07af4ad85f8b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:31:22.776306 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-da1f112b-9546-4627-863e-07af4ad85f8b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:31:22.778408 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1023/2024] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Nov 2 07:31:22 2022] DELETE /identity/v3/projects/0f2be348b8f44fefbc65fb902427463f => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:31:22.782487 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1f112b-9546-4627-863e-07af4ad85f8b admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:31:22.784170 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-da1f112b-9546-4627-863e-07af4ad85f8b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:31:22.799282 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:31:22.812944 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1002/2025] 10.222.0.18 () {58 vars in 1009 bytes} [Wed Nov 2 07:31:22 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:31:22.827891 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb74dc8-b3f8-471a-8716-036aeb13bf54 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:31:22.829317 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-8bb74dc8-b3f8-471a-8716-036aeb13bf54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:31:22.858802 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:31:22.901732 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1f112b-9546-4627-863e-07af4ad85f8b admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:31:22.911514 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c547b84-ce8a-4f97-b5c3-ed015bcad1dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:31:22.912421 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c547b84-ce8a-4f97-b5c3-ed015bcad1dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:22.912853 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c547b84-ce8a-4f97-b5c3-ed015bcad1dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:22.913275 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c547b84-ce8a-4f97-b5c3-ed015bcad1dc admin admin] PATH_INFO: `/v3/users/9099f5b195d54445b92ad14796bc7887` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:22.914799 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c547b84-ce8a-4f97-b5c3-ed015bcad1dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=9099f5b195d54445b92ad14796bc7887)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:31:22.918248 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c547b84-ce8a-4f97-b5c3-ed015bcad1dc admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:31:22.928763 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb74dc8-b3f8-471a-8716-036aeb13bf54 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:31:22.935247 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe86846-ca5e-4a95-b64a-28a7a30064a1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9116f47889f4db39e0203d6cdae1593', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:31:22.936077 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe86846-ca5e-4a95-b64a-28a7a30064a1 service swift] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:22.936533 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe86846-ca5e-4a95-b64a-28a7a30064a1 service swift] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:22.936881 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe86846-ca5e-4a95-b64a-28a7a30064a1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:22.947646 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe86846-ca5e-4a95-b64a-28a7a30064a1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:31:22.949716 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe86846-ca5e-4a95-b64a-28a7a30064a1 service swift] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:31:22.959983 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1003/2026] 10.222.0.18 () {60 vars in 1384 bytes} [Wed Nov 2 07:31:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Nov 02 07:31:23.037076 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-2c547b84-ce8a-4f97-b5c3-ed015bcad1dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9099f5b195d54445b92ad14796bc7887'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:31:23.048932 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-2c547b84-ce8a-4f97-b5c3-ed015bcad1dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9099f5b195d54445b92ad14796bc7887'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:31:23.050457 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-2c547b84-ce8a-4f97-b5c3-ed015bcad1dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9099f5b195d54445b92ad14796bc7887'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:31:23.068382 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-2c547b84-ce8a-4f97-b5c3-ed015bcad1dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9099f5b195d54445b92ad14796bc7887'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:31:23.073475 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-2c547b84-ce8a-4f97-b5c3-ed015bcad1dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9099f5b195d54445b92ad14796bc7887'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:31:23.115181 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1024/2027] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Nov 2 07:31:22 2022] DELETE /identity/v3/users/9099f5b195d54445b92ad14796bc7887 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:31:23.726928 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe86846-ca5e-4a95-b64a-28a7a30064a1 service swift] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:31:23.762155 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe86846-ca5e-4a95-b64a-28a7a30064a1 service swift] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:31:23.849915 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ec5fb0-880b-4158-b692-770eeaf2277b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:31:23.851531 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ec5fb0-880b-4158-b692-770eeaf2277b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:23.852101 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ec5fb0-880b-4158-b692-770eeaf2277b admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:23.852458 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ec5fb0-880b-4158-b692-770eeaf2277b admin admin] PATH_INFO: `/v3/projects/1b4fccca12b44b0ab55dc1f67ee85305` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:23.853677 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ec5fb0-880b-4158-b692-770eeaf2277b admin admin] RBAC: Authorizing `identity:delete_project(project_id=1b4fccca12b44b0ab55dc1f67ee85305)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:31:23.858316 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ec5fb0-880b-4158-b692-770eeaf2277b admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:31:23.917508 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-b6ec5fb0-880b-4158-b692-770eeaf2277b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1b4fccca12b44b0ab55dc1f67ee85305', 'request_id': 'req-b6ec5fb0-880b-4158-b692-770eeaf2277b'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:31:23.927307 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-b6ec5fb0-880b-4158-b692-770eeaf2277b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1b4fccca12b44b0ab55dc1f67ee85305', 'request_id': 'req-b6ec5fb0-880b-4158-b692-770eeaf2277b'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:31:23.937373 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-b6ec5fb0-880b-4158-b692-770eeaf2277b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1b4fccca12b44b0ab55dc1f67ee85305', 'request_id': 'req-b6ec5fb0-880b-4158-b692-770eeaf2277b'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:31:23.955580 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-b6ec5fb0-880b-4158-b692-770eeaf2277b admin admin] The token cache is being invalidate because project 1b4fccca12b44b0ab55dc1f67ee85305 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81501) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:31:23.957978 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-b6ec5fb0-880b-4158-b692-770eeaf2277b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:31:23.960757 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-b6ec5fb0-880b-4158-b692-770eeaf2277b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:31:23.963414 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1004/2028] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Nov 2 07:31:23 2022] DELETE /identity/v3/projects/1b4fccca12b44b0ab55dc1f67ee85305 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:31:23.968080 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c547b84-ce8a-4f97-b5c3-ed015bcad1dc admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:31:23.970210 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-2c547b84-ce8a-4f97-b5c3-ed015bcad1dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:31:23.985761 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c547b84-ce8a-4f97-b5c3-ed015bcad1dc admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:31:24.071188 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50c5afa-4445-4d41-a645-743d92cc401a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:31:24.072696 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50c5afa-4445-4d41-a645-743d92cc401a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:24.073235 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50c5afa-4445-4d41-a645-743d92cc401a admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:24.073734 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50c5afa-4445-4d41-a645-743d92cc401a admin admin] PATH_INFO: `/v3/users/93e457f801114e4db1989a9d6ac3f50d` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:24.076189 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50c5afa-4445-4d41-a645-743d92cc401a admin admin] RBAC: Authorizing `identity:delete_user(user_id=93e457f801114e4db1989a9d6ac3f50d)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:31:24.078436 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50c5afa-4445-4d41-a645-743d92cc401a admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:31:24.142266 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:31:24.172933 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-c50c5afa-4445-4d41-a645-743d92cc401a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '93e457f801114e4db1989a9d6ac3f50d'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:31:24.184617 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-c50c5afa-4445-4d41-a645-743d92cc401a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '93e457f801114e4db1989a9d6ac3f50d'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:31:24.185641 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-c50c5afa-4445-4d41-a645-743d92cc401a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '93e457f801114e4db1989a9d6ac3f50d'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:31:24.197777 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-c50c5afa-4445-4d41-a645-743d92cc401a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '93e457f801114e4db1989a9d6ac3f50d'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:31:24.199519 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-c50c5afa-4445-4d41-a645-743d92cc401a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '93e457f801114e4db1989a9d6ac3f50d'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:31:24.228538 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1025/2029] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Nov 2 07:31:23 2022] DELETE /identity/v3/users/93e457f801114e4db1989a9d6ac3f50d => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 07:31:24.633308 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ec5fb0-880b-4158-b692-770eeaf2277b admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:31:24.635145 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-b6ec5fb0-880b-4158-b692-770eeaf2277b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:31:24.655269 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ec5fb0-880b-4158-b692-770eeaf2277b admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:31:24.724661 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:31:24.769803 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec01ba5e-63d5-4901-966b-2379db3dae81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:31:24.770496 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec01ba5e-63d5-4901-966b-2379db3dae81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:24.770786 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec01ba5e-63d5-4901-966b-2379db3dae81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:24.771222 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec01ba5e-63d5-4901-966b-2379db3dae81 admin admin] PATH_INFO: `/v3/projects/aaef001e3398403c9dc419142795859b` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:24.772493 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec01ba5e-63d5-4901-966b-2379db3dae81 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aaef001e3398403c9dc419142795859b)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:31:24.775341 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec01ba5e-63d5-4901-966b-2379db3dae81 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:31:24.847274 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-ec01ba5e-63d5-4901-966b-2379db3dae81 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aaef001e3398403c9dc419142795859b', 'request_id': 'req-ec01ba5e-63d5-4901-966b-2379db3dae81'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:31:24.855530 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-ec01ba5e-63d5-4901-966b-2379db3dae81 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aaef001e3398403c9dc419142795859b', 'request_id': 'req-ec01ba5e-63d5-4901-966b-2379db3dae81'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:31:24.881889 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-ec01ba5e-63d5-4901-966b-2379db3dae81 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aaef001e3398403c9dc419142795859b', 'request_id': 'req-ec01ba5e-63d5-4901-966b-2379db3dae81'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:31:24.894726 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-ec01ba5e-63d5-4901-966b-2379db3dae81 admin admin] The token cache is being invalidate because project aaef001e3398403c9dc419142795859b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81501) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:31:24.895016 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-ec01ba5e-63d5-4901-966b-2379db3dae81 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:31:24.895673 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-ec01ba5e-63d5-4901-966b-2379db3dae81 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:31:24.899208 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1005/2030] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Nov 2 07:31:24 2022] DELETE /identity/v3/projects/aaef001e3398403c9dc419142795859b => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:31:24.907106 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c01cd0-f5b7-4246-9ee6-a44b0af7e430 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:24.907691 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c01cd0-f5b7-4246-9ee6-a44b0af7e430 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:24.908154 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c01cd0-f5b7-4246-9ee6-a44b0af7e430 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:24.930897 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-c0c01cd0-f5b7-4246-9ee6-a44b0af7e430 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:31:24.939146 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-c0c01cd0-f5b7-4246-9ee6-a44b0af7e430 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:31:25.009967 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1026/2031] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:31:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:31:25.019120 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018df74e-5a7e-495d-9068-95b8b087e189 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:25.019415 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018df74e-5a7e-495d-9068-95b8b087e189 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:25.019773 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018df74e-5a7e-495d-9068-95b8b087e189 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:25.041732 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-018df74e-5a7e-495d-9068-95b8b087e189 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:31:25.043023 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-018df74e-5a7e-495d-9068-95b8b087e189 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:31:25.060635 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1006/2032] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:31:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:31:25.065024 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c01cd0-f5b7-4246-9ee6-a44b0af7e430 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:31:25.076029 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c01cd0-f5b7-4246-9ee6-a44b0af7e430 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:31:25.080912 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344c2418-7a4e-4b2f-956d-677e6236b783 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:31:25.082868 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344c2418-7a4e-4b2f-956d-677e6236b783 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:25.083815 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344c2418-7a4e-4b2f-956d-677e6236b783 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:25.084226 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344c2418-7a4e-4b2f-956d-677e6236b783 admin admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:25.085064 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344c2418-7a4e-4b2f-956d-677e6236b783 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:31:25.085671 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344c2418-7a4e-4b2f-956d-677e6236b783 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:31:25.087794 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344c2418-7a4e-4b2f-956d-677e6236b783 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:31:25.102403 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1027/2033] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Nov 2 07:31:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:31:25.105506 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018df74e-5a7e-495d-9068-95b8b087e189 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:31:25.115769 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018df74e-5a7e-495d-9068-95b8b087e189 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:31:25.121034 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd14a9b0-22f2-4367-85d6-3815590c41a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:31:25.121630 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd14a9b0-22f2-4367-85d6-3815590c41a1 admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:25.121911 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd14a9b0-22f2-4367-85d6-3815590c41a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:25.122185 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd14a9b0-22f2-4367-85d6-3815590c41a1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:25.125035 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd14a9b0-22f2-4367-85d6-3815590c41a1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:31:25.125035 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd14a9b0-22f2-4367-85d6-3815590c41a1 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:31:25.160966 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1007/2034] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Nov 2 07:31:25 2022] POST /identity/v3/projects => generated 371 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:31:25.164138 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344c2418-7a4e-4b2f-956d-677e6236b783 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:31:25.175495 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344c2418-7a4e-4b2f-956d-677e6236b783 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:31:25.264134 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6e6f38-8bd7-4aa5-b44d-0cda613498e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:31:25.265055 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6e6f38-8bd7-4aa5-b44d-0cda613498e2 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:25.265456 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6e6f38-8bd7-4aa5-b44d-0cda613498e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:25.265827 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6e6f38-8bd7-4aa5-b44d-0cda613498e2 admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:25.266605 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6e6f38-8bd7-4aa5-b44d-0cda613498e2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:31:25.269267 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6e6f38-8bd7-4aa5-b44d-0cda613498e2 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:31:25.292497 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1028/2035] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Nov 2 07:31:25 2022] POST /identity/v3/users => generated 286 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:31:25.298112 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd14a9b0-22f2-4367-85d6-3815590c41a1 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:31:25.309300 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd14a9b0-22f2-4367-85d6-3815590c41a1 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:31:25.316490 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fa23ba-e036-44c6-878d-1365d5bf3bd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:31:25.316711 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fa23ba-e036-44c6-878d-1365d5bf3bd9 admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:25.317058 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fa23ba-e036-44c6-878d-1365d5bf3bd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:25.317474 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fa23ba-e036-44c6-878d-1365d5bf3bd9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:25.318680 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fa23ba-e036-44c6-878d-1365d5bf3bd9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:31:25.320813 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fa23ba-e036-44c6-878d-1365d5bf3bd9 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:31:25.331935 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1008/2036] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:31:25 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:31:25.336430 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6e6f38-8bd7-4aa5-b44d-0cda613498e2 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:31:25.349084 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6e6f38-8bd7-4aa5-b44d-0cda613498e2 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:31:25.354072 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f3ef04-9046-43b9-8052-e345f9cfc4ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:31:25.354717 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f3ef04-9046-43b9-8052-e345f9cfc4ad admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:25.355008 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f3ef04-9046-43b9-8052-e345f9cfc4ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:25.359066 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f3ef04-9046-43b9-8052-e345f9cfc4ad admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:25.359519 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f3ef04-9046-43b9-8052-e345f9cfc4ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:31:25.361199 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f3ef04-9046-43b9-8052-e345f9cfc4ad admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:31:25.382620 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1029/2037] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:31:25 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:31:25.395292 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fa23ba-e036-44c6-878d-1365d5bf3bd9 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:31:25.405745 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fa23ba-e036-44c6-878d-1365d5bf3bd9 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:31:25.413290 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd33135f-8e1b-4631-bef9-6499168706cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:31:25.414275 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd33135f-8e1b-4631-bef9-6499168706cc admin admin] REQUEST_METHOD: `PUT` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:25.414756 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd33135f-8e1b-4631-bef9-6499168706cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:25.415037 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd33135f-8e1b-4631-bef9-6499168706cc admin admin] PATH_INFO: `/v3/projects/0c3c0a5719274bfb9f8f557162bf5b4d/users/8cd1d5eadd6b4ca69add9b1186fc76ae/roles/b494fedd00f24f0b84a529e8672a156a` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:25.433832 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd33135f-8e1b-4631-bef9-6499168706cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c3c0a5719274bfb9f8f557162bf5b4d, user_id=8cd1d5eadd6b4ca69add9b1186fc76ae, role_id=b494fedd00f24f0b84a529e8672a156a)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:31:25.438517 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd33135f-8e1b-4631-bef9-6499168706cc admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:31:25.458412 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1009/2038] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Nov 2 07:31:25 2022] PUT /identity/v3/projects/0c3c0a5719274bfb9f8f557162bf5b4d/users/8cd1d5eadd6b4ca69add9b1186fc76ae/roles/b494fedd00f24f0b84a529e8672a156a => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:31:25.474323 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed0c308-4128-4fbb-9f70-e9633d626b3d None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:25.474881 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed0c308-4128-4fbb-9f70-e9633d626b3d None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:25.475220 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed0c308-4128-4fbb-9f70-e9633d626b3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:25.515982 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:31:25.531320 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-4ed0c308-4128-4fbb-9f70-e9633d626b3d None None] MFA Rules not processed for user `8cd1d5eadd6b4ca69add9b1186fc76ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:31:25.533925 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-4ed0c308-4128-4fbb-9f70-e9633d626b3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:31:25.662977 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1030/2039] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:31:25 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:31:25.675645 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd33135f-8e1b-4631-bef9-6499168706cc admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:31:25.677470 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-fd33135f-8e1b-4631-bef9-6499168706cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:31:25.753292 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd33135f-8e1b-4631-bef9-6499168706cc admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:31:25.757301 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c74d76-d7d8-4dc9-a1e1-474d69d8ffc5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43bb326fb82e4d2c83fb2eaf91860dc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:31:25.757918 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c74d76-d7d8-4dc9-a1e1-474d69d8ffc5 service cinder] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:25.758223 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c74d76-d7d8-4dc9-a1e1-474d69d8ffc5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:25.758494 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c74d76-d7d8-4dc9-a1e1-474d69d8ffc5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:25.772744 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c74d76-d7d8-4dc9-a1e1-474d69d8ffc5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:31:25.774819 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c74d76-d7d8-4dc9-a1e1-474d69d8ffc5 service cinder] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:31:25.785360 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1010/2040] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:31:25 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:31:25.812095 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed0c308-4128-4fbb-9f70-e9633d626b3d None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:31:25.958588 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed0c308-4128-4fbb-9f70-e9633d626b3d None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:31:25.963511 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b94f72-9086-4bf1-9d06-9d504f8aef76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:31:25.964201 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b94f72-9086-4bf1-9d06-9d504f8aef76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:25.964745 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b94f72-9086-4bf1-9d06-9d504f8aef76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:25.965022 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b94f72-9086-4bf1-9d06-9d504f8aef76 admin admin] PATH_INFO: `/v3/users/8cd1d5eadd6b4ca69add9b1186fc76ae` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:25.967026 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b94f72-9086-4bf1-9d06-9d504f8aef76 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8cd1d5eadd6b4ca69add9b1186fc76ae)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:31:25.968992 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b94f72-9086-4bf1-9d06-9d504f8aef76 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:31:26.040134 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-d0b94f72-9086-4bf1-9d06-9d504f8aef76 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8cd1d5eadd6b4ca69add9b1186fc76ae'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:31:26.054403 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-d0b94f72-9086-4bf1-9d06-9d504f8aef76 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8cd1d5eadd6b4ca69add9b1186fc76ae'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:31:26.056683 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-d0b94f72-9086-4bf1-9d06-9d504f8aef76 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8cd1d5eadd6b4ca69add9b1186fc76ae'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:31:26.071006 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-d0b94f72-9086-4bf1-9d06-9d504f8aef76 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8cd1d5eadd6b4ca69add9b1186fc76ae'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:31:26.072373 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-d0b94f72-9086-4bf1-9d06-9d504f8aef76 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8cd1d5eadd6b4ca69add9b1186fc76ae'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:31:26.110421 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1031/2041] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Nov 2 07:31:25 2022] DELETE /identity/v3/users/8cd1d5eadd6b4ca69add9b1186fc76ae => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:31:26.112776 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8e089c-e048-4a34-a9af-d2634959feed None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:26.113258 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8e089c-e048-4a34-a9af-d2634959feed None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:26.114292 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8e089c-e048-4a34-a9af-d2634959feed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:26.135529 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-ba8e089c-e048-4a34-a9af-d2634959feed None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:31:26.136811 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-ba8e089c-e048-4a34-a9af-d2634959feed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:31:26.142875 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:31:26.221673 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1011/2042] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:31:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:31:26.234641 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b94f72-9086-4bf1-9d06-9d504f8aef76 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:31:26.236876 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-d0b94f72-9086-4bf1-9d06-9d504f8aef76 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:31:26.266296 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:31:26.334887 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b94f72-9086-4bf1-9d06-9d504f8aef76 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:31:26.340340 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529de3a4-194b-4f15-b745-336127590bbc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3ec6ecb85f94ef59762877d9ea33ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:31:26.340967 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529de3a4-194b-4f15-b745-336127590bbc service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:26.341302 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529de3a4-194b-4f15-b745-336127590bbc service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:26.341633 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529de3a4-194b-4f15-b745-336127590bbc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:26.360346 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529de3a4-194b-4f15-b745-336127590bbc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:31:26.363077 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529de3a4-194b-4f15-b745-336127590bbc service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:31:26.380374 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1032/2043] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:31:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:31:27.429233 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8e089c-e048-4a34-a9af-d2634959feed None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:31:27.432166 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-ba8e089c-e048-4a34-a9af-d2634959feed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:31:27.454784 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8e089c-e048-4a34-a9af-d2634959feed None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:31:27.463010 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010fbc09-526e-423a-a5de-fcec32587659 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:31:27.464081 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010fbc09-526e-423a-a5de-fcec32587659 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:27.464482 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010fbc09-526e-423a-a5de-fcec32587659 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:27.464844 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010fbc09-526e-423a-a5de-fcec32587659 admin admin] PATH_INFO: `/v3/projects/0c3c0a5719274bfb9f8f557162bf5b4d` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:27.466204 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010fbc09-526e-423a-a5de-fcec32587659 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0c3c0a5719274bfb9f8f557162bf5b4d)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:31:27.470645 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010fbc09-526e-423a-a5de-fcec32587659 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:31:27.553096 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-010fbc09-526e-423a-a5de-fcec32587659 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0c3c0a5719274bfb9f8f557162bf5b4d', 'request_id': 'req-010fbc09-526e-423a-a5de-fcec32587659'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:31:27.572826 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-010fbc09-526e-423a-a5de-fcec32587659 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0c3c0a5719274bfb9f8f557162bf5b4d', 'request_id': 'req-010fbc09-526e-423a-a5de-fcec32587659'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:31:27.583895 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-010fbc09-526e-423a-a5de-fcec32587659 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0c3c0a5719274bfb9f8f557162bf5b4d', 'request_id': 'req-010fbc09-526e-423a-a5de-fcec32587659'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:31:27.603743 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-010fbc09-526e-423a-a5de-fcec32587659 admin admin] The token cache is being invalidate because project 0c3c0a5719274bfb9f8f557162bf5b4d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81501) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:31:27.604095 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-010fbc09-526e-423a-a5de-fcec32587659 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:31:27.604859 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-010fbc09-526e-423a-a5de-fcec32587659 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:31:27.613772 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1012/2044] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Nov 2 07:31:27 2022] DELETE /identity/v3/projects/0c3c0a5719274bfb9f8f557162bf5b4d => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:31:27.614952 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e800c229-f3f2-43bc-a173-d67aa18a46e6 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:27.617138 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e800c229-f3f2-43bc-a173-d67aa18a46e6 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:27.617791 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e800c229-f3f2-43bc-a173-d67aa18a46e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:27.642436 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-e800c229-f3f2-43bc-a173-d67aa18a46e6 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:31:27.648765 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-e800c229-f3f2-43bc-a173-d67aa18a46e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:31:27.815018 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1033/2045] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:31:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:31:27.818553 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c921ba-2e8d-414e-8ede-007fde4cdd33 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:27.819082 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c921ba-2e8d-414e-8ede-007fde4cdd33 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:27.819564 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c921ba-2e8d-414e-8ede-007fde4cdd33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:27.851642 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-f8c921ba-2e8d-414e-8ede-007fde4cdd33 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:31:27.853061 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-f8c921ba-2e8d-414e-8ede-007fde4cdd33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:31:27.880570 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1013/2046] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:31:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:31:27.880935 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e800c229-f3f2-43bc-a173-d67aa18a46e6 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:31:27.900996 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e800c229-f3f2-43bc-a173-d67aa18a46e6 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:31:27.915113 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7221db2-e21a-4176-b6e2-bbf321811f53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:31:27.915749 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7221db2-e21a-4176-b6e2-bbf321811f53 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:27.916048 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7221db2-e21a-4176-b6e2-bbf321811f53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:27.916300 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7221db2-e21a-4176-b6e2-bbf321811f53 admin admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:27.916958 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7221db2-e21a-4176-b6e2-bbf321811f53 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:31:27.917260 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7221db2-e21a-4176-b6e2-bbf321811f53 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:31:27.920529 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7221db2-e21a-4176-b6e2-bbf321811f53 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:31:27.940474 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1034/2047] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Nov 2 07:31:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:31:27.944898 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c921ba-2e8d-414e-8ede-007fde4cdd33 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:31:27.955785 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c921ba-2e8d-414e-8ede-007fde4cdd33 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:31:27.960996 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5304a39f-4163-4368-a109-00a4f4f05847 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:31:27.961634 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5304a39f-4163-4368-a109-00a4f4f05847 admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:27.962002 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5304a39f-4163-4368-a109-00a4f4f05847 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:27.962262 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5304a39f-4163-4368-a109-00a4f4f05847 admin admin] PATH_INFO: `/v3/projects` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:27.962922 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5304a39f-4163-4368-a109-00a4f4f05847 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:31:27.964807 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5304a39f-4163-4368-a109-00a4f4f05847 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:31:28.012746 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1014/2048] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Nov 2 07:31:27 2022] POST /identity/v3/projects => generated 373 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:31:28.017221 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7221db2-e21a-4176-b6e2-bbf321811f53 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:31:28.033599 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7221db2-e21a-4176-b6e2-bbf321811f53 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:31:28.132964 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e2062e-42b2-4a6a-9791-f75f13cf83c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:31:28.133950 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e2062e-42b2-4a6a-9791-f75f13cf83c4 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:28.134380 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e2062e-42b2-4a6a-9791-f75f13cf83c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:28.134775 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e2062e-42b2-4a6a-9791-f75f13cf83c4 admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:28.135549 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e2062e-42b2-4a6a-9791-f75f13cf83c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:31:28.139547 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e2062e-42b2-4a6a-9791-f75f13cf83c4 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:31:28.185669 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1035/2049] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Nov 2 07:31:28 2022] POST /identity/v3/users => generated 287 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 07:31:28.209801 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5304a39f-4163-4368-a109-00a4f4f05847 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:31:28.240356 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5304a39f-4163-4368-a109-00a4f4f05847 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:31:28.271610 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7672055-e6fd-4f58-bbc4-fca71c4e9708 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:31:28.272300 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7672055-e6fd-4f58-bbc4-fca71c4e9708 admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:28.272583 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7672055-e6fd-4f58-bbc4-fca71c4e9708 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:28.272852 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7672055-e6fd-4f58-bbc4-fca71c4e9708 admin admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:28.276969 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7672055-e6fd-4f58-bbc4-fca71c4e9708 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:31:28.288420 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7672055-e6fd-4f58-bbc4-fca71c4e9708 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:31:28.318552 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1015/2050] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:31:28 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:31:28.321255 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e2062e-42b2-4a6a-9791-f75f13cf83c4 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:31:28.334063 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e2062e-42b2-4a6a-9791-f75f13cf83c4 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:31:28.344813 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc938cf-d623-40e8-8de7-5428d122acee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:31:28.348312 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc938cf-d623-40e8-8de7-5428d122acee admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:28.351229 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc938cf-d623-40e8-8de7-5428d122acee admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:28.351754 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc938cf-d623-40e8-8de7-5428d122acee admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:28.352659 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc938cf-d623-40e8-8de7-5428d122acee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:31:28.354589 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc938cf-d623-40e8-8de7-5428d122acee admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:31:28.370233 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1036/2051] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:31:28 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:31:28.375631 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7672055-e6fd-4f58-bbc4-fca71c4e9708 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:31:28.389654 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7672055-e6fd-4f58-bbc4-fca71c4e9708 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:31:28.396039 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4a3cc2-041e-4a06-8603-c7d4429eac7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:31:28.396947 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4a3cc2-041e-4a06-8603-c7d4429eac7b admin admin] REQUEST_METHOD: `PUT` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:28.397430 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4a3cc2-041e-4a06-8603-c7d4429eac7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:28.397893 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4a3cc2-041e-4a06-8603-c7d4429eac7b admin admin] PATH_INFO: `/v3/projects/82915a36bbdb4b5998fd98833e234ad2/users/040b847b0db549a2a865deb7be6a8da7/roles/b494fedd00f24f0b84a529e8672a156a` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:28.420021 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4a3cc2-041e-4a06-8603-c7d4429eac7b admin admin] RBAC: Authorizing `identity:create_grant(project_id=82915a36bbdb4b5998fd98833e234ad2, user_id=040b847b0db549a2a865deb7be6a8da7, role_id=b494fedd00f24f0b84a529e8672a156a)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:31:28.422801 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4a3cc2-041e-4a06-8603-c7d4429eac7b admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:31:28.449103 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1016/2052] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Nov 2 07:31:28 2022] PUT /identity/v3/projects/82915a36bbdb4b5998fd98833e234ad2/users/040b847b0db549a2a865deb7be6a8da7/roles/b494fedd00f24f0b84a529e8672a156a => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:31:28.455152 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7ae352-8efd-423a-95ef-2b3e23a97d66 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:28.456038 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7ae352-8efd-423a-95ef-2b3e23a97d66 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:28.456495 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7ae352-8efd-423a-95ef-2b3e23a97d66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:28.480791 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-df7ae352-8efd-423a-95ef-2b3e23a97d66 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:31:28.482721 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-df7ae352-8efd-423a-95ef-2b3e23a97d66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:31:28.509928 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:31:28.605045 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1037/2053] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:31:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:31:28.613034 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4a3cc2-041e-4a06-8603-c7d4429eac7b admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:31:28.614917 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-3c4a3cc2-041e-4a06-8603-c7d4429eac7b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:31:28.678622 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:31:28.715805 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4a3cc2-041e-4a06-8603-c7d4429eac7b admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:31:28.720561 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d45574-f890-4eb5-bdab-56addab5a7fa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3ec6ecb85f94ef59762877d9ea33ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:31:28.721264 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d45574-f890-4eb5-bdab-56addab5a7fa service neutron] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:28.721603 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d45574-f890-4eb5-bdab-56addab5a7fa service neutron] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:28.721818 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d45574-f890-4eb5-bdab-56addab5a7fa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:28.742267 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d45574-f890-4eb5-bdab-56addab5a7fa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:31:28.743817 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d45574-f890-4eb5-bdab-56addab5a7fa service neutron] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:31:28.757205 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1017/2054] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:31:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:31:33.443235 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045c046c-b879-4d2b-ba96-f2e10a9bd98b None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:33.443819 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045c046c-b879-4d2b-ba96-f2e10a9bd98b None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:33.443819 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045c046c-b879-4d2b-ba96-f2e10a9bd98b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:33.480132 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-045c046c-b879-4d2b-ba96-f2e10a9bd98b None None] MFA Rules not processed for user `0e571b0f4d6f4fafb71939f501b8deec`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:31:33.481059 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-045c046c-b879-4d2b-ba96-f2e10a9bd98b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:31:33.594408 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1038/2055] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:31:33 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:31:35.086456 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d45574-f890-4eb5-bdab-56addab5a7fa service neutron] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:31:35.098439 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d45574-f890-4eb5-bdab-56addab5a7fa service neutron] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:31:35.117278 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b785bb03-09f2-4e0e-8342-d1e0f7fd449e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3ec6ecb85f94ef59762877d9ea33ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:31:35.118251 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b785bb03-09f2-4e0e-8342-d1e0f7fd449e service neutron] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:35.118721 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b785bb03-09f2-4e0e-8342-d1e0f7fd449e service neutron] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:35.119161 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b785bb03-09f2-4e0e-8342-d1e0f7fd449e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:35.125737 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-b785bb03-09f2-4e0e-8342-d1e0f7fd449e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:31:35.223584 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b785bb03-09f2-4e0e-8342-d1e0f7fd449e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:31:35.231406 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b785bb03-09f2-4e0e-8342-d1e0f7fd449e service neutron] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:31:35.274711 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1018/2056] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:31:35 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:31:36.678182 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7810e612-8187-4edf-97da-453606895305 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:36.678676 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7810e612-8187-4edf-97da-453606895305 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:36.678756 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7810e612-8187-4edf-97da-453606895305 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:36.724989 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-7810e612-8187-4edf-97da-453606895305 None None] MFA Rules not processed for user `040b847b0db549a2a865deb7be6a8da7`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:31:36.726716 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-7810e612-8187-4edf-97da-453606895305 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:31:36.845739 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1039/2057] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:31:36 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:31:36.847172 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b785bb03-09f2-4e0e-8342-d1e0f7fd449e service neutron] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:31:36.868363 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b785bb03-09f2-4e0e-8342-d1e0f7fd449e service neutron] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:31:36.875506 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70e9ce5-7e3a-42fc-8beb-f8a495cd3eed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:31:36.876986 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70e9ce5-7e3a-42fc-8beb-f8a495cd3eed admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:36.877416 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70e9ce5-7e3a-42fc-8beb-f8a495cd3eed admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:36.877826 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70e9ce5-7e3a-42fc-8beb-f8a495cd3eed admin admin] PATH_INFO: `/v3/projects` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:36.878821 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70e9ce5-7e3a-42fc-8beb-f8a495cd3eed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:31:36.880992 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70e9ce5-7e3a-42fc-8beb-f8a495cd3eed admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:31:36.932593 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1019/2058] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Nov 2 07:31:36 2022] POST /identity/v3/projects => generated 373 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:31:36.938183 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7810e612-8187-4edf-97da-453606895305 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:31:36.956159 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7810e612-8187-4edf-97da-453606895305 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:31:37.125272 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500399b1-6072-4891-b279-770afc3b9cc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:31:37.125845 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500399b1-6072-4891-b279-770afc3b9cc2 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:37.128896 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500399b1-6072-4891-b279-770afc3b9cc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:37.129460 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500399b1-6072-4891-b279-770afc3b9cc2 admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:37.132655 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500399b1-6072-4891-b279-770afc3b9cc2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:31:37.148609 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500399b1-6072-4891-b279-770afc3b9cc2 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:31:37.179637 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:31:37.211141 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1040/2059] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Nov 2 07:31:36 2022] POST /identity/v3/users => generated 293 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 07:31:37.212692 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70e9ce5-7e3a-42fc-8beb-f8a495cd3eed admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:31:37.227020 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70e9ce5-7e3a-42fc-8beb-f8a495cd3eed admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:31:37.249514 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d13726-49ea-4c88-bbca-3e585dd8a695 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:31:37.252036 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d13726-49ea-4c88-bbca-3e585dd8a695 admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:37.252389 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d13726-49ea-4c88-bbca-3e585dd8a695 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:37.252824 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d13726-49ea-4c88-bbca-3e585dd8a695 admin admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:37.254461 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d13726-49ea-4c88-bbca-3e585dd8a695 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:31:37.258565 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d13726-49ea-4c88-bbca-3e585dd8a695 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:31:37.292998 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500399b1-6072-4891-b279-770afc3b9cc2 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:31:37.298266 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1020/2060] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:31:37 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:31:37.306894 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500399b1-6072-4891-b279-770afc3b9cc2 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:31:37.320235 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478d2702-d5cd-4e2c-8ba5-e86925c48420 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:31:37.321017 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478d2702-d5cd-4e2c-8ba5-e86925c48420 admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:37.321367 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478d2702-d5cd-4e2c-8ba5-e86925c48420 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:37.321718 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478d2702-d5cd-4e2c-8ba5-e86925c48420 admin admin] PATH_INFO: `/v3/projects/d69c92f22f8d42bdb324ed5d021b56e3/users/7fd3e52fad0f4ff9af317adaa3e1aba4/roles/c4bf019fc6c44c71a6c984f6d0bab75f` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:37.360181 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478d2702-d5cd-4e2c-8ba5-e86925c48420 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d69c92f22f8d42bdb324ed5d021b56e3, user_id=7fd3e52fad0f4ff9af317adaa3e1aba4, role_id=c4bf019fc6c44c71a6c984f6d0bab75f)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:31:37.363253 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478d2702-d5cd-4e2c-8ba5-e86925c48420 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:31:37.426978 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1041/2061] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Nov 2 07:31:37 2022] PUT /identity/v3/projects/d69c92f22f8d42bdb324ed5d021b56e3/users/7fd3e52fad0f4ff9af317adaa3e1aba4/roles/c4bf019fc6c44c71a6c984f6d0bab75f => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:31:37.431182 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d13726-49ea-4c88-bbca-3e585dd8a695 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:31:37.452276 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d13726-49ea-4c88-bbca-3e585dd8a695 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:31:37.727485 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4069e26-c585-45e2-a5b4-491b8db783c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:31:37.728836 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4069e26-c585-45e2-a5b4-491b8db783c6 admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:37.729153 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4069e26-c585-45e2-a5b4-491b8db783c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:37.729711 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4069e26-c585-45e2-a5b4-491b8db783c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:37.730459 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4069e26-c585-45e2-a5b4-491b8db783c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:31:37.733627 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4069e26-c585-45e2-a5b4-491b8db783c6 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:31:37.767951 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1021/2062] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:31:37 2022] GET /identity/v3/roles => generated 1420 bytes in 337 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:31:37.772252 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478d2702-d5cd-4e2c-8ba5-e86925c48420 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:31:37.838470 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478d2702-d5cd-4e2c-8ba5-e86925c48420 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:31:37.845968 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bffd426-39d1-41db-837c-92fe3a0d5f7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:31:37.846939 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bffd426-39d1-41db-837c-92fe3a0d5f7e admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:37.848629 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bffd426-39d1-41db-837c-92fe3a0d5f7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:37.849084 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bffd426-39d1-41db-837c-92fe3a0d5f7e admin admin] PATH_INFO: `/v3/projects/d69c92f22f8d42bdb324ed5d021b56e3/users/7fd3e52fad0f4ff9af317adaa3e1aba4/roles/c4bf019fc6c44c71a6c984f6d0bab75f` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:37.852471 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bffd426-39d1-41db-837c-92fe3a0d5f7e admin admin] RBAC: Authorizing `identity:create_grant(project_id=d69c92f22f8d42bdb324ed5d021b56e3, user_id=7fd3e52fad0f4ff9af317adaa3e1aba4, role_id=c4bf019fc6c44c71a6c984f6d0bab75f)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:31:37.856568 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bffd426-39d1-41db-837c-92fe3a0d5f7e admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:31:37.874391 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1042/2063] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Nov 2 07:31:37 2022] PUT /identity/v3/projects/d69c92f22f8d42bdb324ed5d021b56e3/users/7fd3e52fad0f4ff9af317adaa3e1aba4/roles/c4bf019fc6c44c71a6c984f6d0bab75f => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:31:45.096130 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd32ffbb-a57c-4e2b-ad00-bb901640663f None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:45.097099 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd32ffbb-a57c-4e2b-ad00-bb901640663f None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:45.097384 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd32ffbb-a57c-4e2b-ad00-bb901640663f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:45.131031 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-cd32ffbb-a57c-4e2b-ad00-bb901640663f None None] MFA Rules not processed for user `7fd3e52fad0f4ff9af317adaa3e1aba4`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:31:45.132527 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-cd32ffbb-a57c-4e2b-ad00-bb901640663f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:31:45.155838 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:31:45.218923 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1022/2064] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:31:45 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:31:45.233754 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bffd426-39d1-41db-837c-92fe3a0d5f7e admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:31:45.235534 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-0bffd426-39d1-41db-837c-92fe3a0d5f7e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:31:45.315529 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bffd426-39d1-41db-837c-92fe3a0d5f7e admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:31:45.321010 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fe1051-1a36-4e5d-a680-39ffa34b5111 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e48cba3f804cf08b2477f01ae95179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:31:45.321623 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fe1051-1a36-4e5d-a680-39ffa34b5111 service nova] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:45.321901 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fe1051-1a36-4e5d-a680-39ffa34b5111 service nova] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:45.322174 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fe1051-1a36-4e5d-a680-39ffa34b5111 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:45.390139 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fe1051-1a36-4e5d-a680-39ffa34b5111 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:31:45.391803 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fe1051-1a36-4e5d-a680-39ffa34b5111 service nova] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:31:45.403291 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1043/2065] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:31:45 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:31:46.723957 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173f38b1-cff3-4517-a5e7-8d473f4827fc None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:46.724795 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173f38b1-cff3-4517-a5e7-8d473f4827fc None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:46.725216 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173f38b1-cff3-4517-a5e7-8d473f4827fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:46.758469 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-173f38b1-cff3-4517-a5e7-8d473f4827fc None None] MFA Rules not processed for user `040b847b0db549a2a865deb7be6a8da7`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:31:46.759891 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-173f38b1-cff3-4517-a5e7-8d473f4827fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:31:46.770478 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1023/2066] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:31:46 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:31:49.079455 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec43b6c5-cf88-41b5-ac5c-949ae08358fd None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:49.080029 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec43b6c5-cf88-41b5-ac5c-949ae08358fd None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:49.080029 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec43b6c5-cf88-41b5-ac5c-949ae08358fd None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:49.082096 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1044/2067] 10.222.0.18 () {54 vars in 921 bytes} [Wed Nov 2 07:31:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:31:49.088757 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae4c4e6-2a94-4789-b100-c5e5cde3828b None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:49.089658 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae4c4e6-2a94-4789-b100-c5e5cde3828b None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:49.089930 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae4c4e6-2a94-4789-b100-c5e5cde3828b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:49.125072 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-6ae4c4e6-2a94-4789-b100-c5e5cde3828b None None] MFA Rules not processed for user `69e48cba3f804cf08b2477f01ae95179`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:31:49.126462 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-6ae4c4e6-2a94-4789-b100-c5e5cde3828b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:31:49.137684 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1024/2068] 10.222.0.18 () {58 vars in 1018 bytes} [Wed Nov 2 07:31:49 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:31:49.156836 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec43b6c5-cf88-41b5-ac5c-949ae08358fd None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:31:49.158867 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-ec43b6c5-cf88-41b5-ac5c-949ae08358fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:31:49.254205 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec43b6c5-cf88-41b5-ac5c-949ae08358fd None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:31:49.258328 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5086720c-4169-4f41-8798-ae144fdff53d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5353f2af0f44d668e2a269647aa1dc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:31:49.259297 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5086720c-4169-4f41-8798-ae144fdff53d service glance] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:49.259676 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5086720c-4169-4f41-8798-ae144fdff53d service glance] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:49.260093 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5086720c-4169-4f41-8798-ae144fdff53d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:49.280086 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:31:49.280086 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5086720c-4169-4f41-8798-ae144fdff53d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:31:49.280086 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5086720c-4169-4f41-8798-ae144fdff53d service glance] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:31:49.294032 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1045/2069] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:31:49 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:31:49.384140 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a553e45-e539-44e7-af63-b6fb8c8d45c1 None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:49.384808 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a553e45-e539-44e7-af63-b6fb8c8d45c1 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:49.385339 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a553e45-e539-44e7-af63-b6fb8c8d45c1 None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:49.390385 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1025/2070] 10.222.0.18 () {54 vars in 921 bytes} [Wed Nov 2 07:31:49 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:31:49.398845 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cca9857-c5a5-44b7-bddd-459fd6e5b7df None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:49.399138 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cca9857-c5a5-44b7-bddd-459fd6e5b7df None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:49.399422 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cca9857-c5a5-44b7-bddd-459fd6e5b7df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:49.427407 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-2cca9857-c5a5-44b7-bddd-459fd6e5b7df None None] MFA Rules not processed for user `b3ec6ecb85f94ef59762877d9ea33ed0`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:31:49.428621 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-2cca9857-c5a5-44b7-bddd-459fd6e5b7df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:31:49.492167 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1046/2071] 10.222.0.18 () {58 vars in 1018 bytes} [Wed Nov 2 07:31:49 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:31:49.504884 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a553e45-e539-44e7-af63-b6fb8c8d45c1 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:31:49.506582 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-8a553e45-e539-44e7-af63-b6fb8c8d45c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:31:49.529830 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a553e45-e539-44e7-af63-b6fb8c8d45c1 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:31:49.536197 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5aea1e-15c6-4498-9e5d-86d38a2ee42f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3ec6ecb85f94ef59762877d9ea33ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:31:49.537107 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5aea1e-15c6-4498-9e5d-86d38a2ee42f service neutron] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:49.537789 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5aea1e-15c6-4498-9e5d-86d38a2ee42f service neutron] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:49.538131 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5aea1e-15c6-4498-9e5d-86d38a2ee42f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:49.547644 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5aea1e-15c6-4498-9e5d-86d38a2ee42f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:31:49.549087 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5aea1e-15c6-4498-9e5d-86d38a2ee42f service neutron] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:31:49.558029 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1026/2072] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:31:49 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:31:49.662014 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184f22b3-fecd-404c-bef9-149c52210b64 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:49.662445 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184f22b3-fecd-404c-bef9-149c52210b64 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:49.662844 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184f22b3-fecd-404c-bef9-149c52210b64 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:49.664542 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1047/2073] 10.222.0.18 () {54 vars in 921 bytes} [Wed Nov 2 07:31:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:31:49.671654 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7503408d-15d9-4c3f-9a69-50a8df19c0d3 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:49.672239 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7503408d-15d9-4c3f-9a69-50a8df19c0d3 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:49.672621 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7503408d-15d9-4c3f-9a69-50a8df19c0d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:49.709412 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-7503408d-15d9-4c3f-9a69-50a8df19c0d3 None None] MFA Rules not processed for user `69e48cba3f804cf08b2477f01ae95179`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:31:49.710713 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-7503408d-15d9-4c3f-9a69-50a8df19c0d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:31:49.721634 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1027/2074] 10.222.0.18 () {58 vars in 1018 bytes} [Wed Nov 2 07:31:49 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:31:49.740641 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184f22b3-fecd-404c-bef9-149c52210b64 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:31:49.764078 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184f22b3-fecd-404c-bef9-149c52210b64 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:31:49.767774 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24363589-e7b6-41b1-a3d4-18c6f58ee27d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3ec6ecb85f94ef59762877d9ea33ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:31:49.768371 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24363589-e7b6-41b1-a3d4-18c6f58ee27d service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:49.777126 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24363589-e7b6-41b1-a3d4-18c6f58ee27d service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:49.777126 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24363589-e7b6-41b1-a3d4-18c6f58ee27d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:49.787859 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e44011-d915-47dc-a92b-cb4b1610588e None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:49.788081 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e44011-d915-47dc-a92b-cb4b1610588e None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:49.788359 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e44011-d915-47dc-a92b-cb4b1610588e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:49.797789 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24363589-e7b6-41b1-a3d4-18c6f58ee27d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:31:49.799404 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24363589-e7b6-41b1-a3d4-18c6f58ee27d service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:31:49.808322 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-50e44011-d915-47dc-a92b-cb4b1610588e None None] MFA Rules not processed for user `0b55941561934c818a2b5eb2c9a19d22`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:31:49.809539 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-50e44011-d915-47dc-a92b-cb4b1610588e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:31:49.811320 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1048/2075] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:31:49 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:31:49.870996 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1028/2076] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:31:49 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:31:54.654670 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfb1b4f-6ff5-4b68-953b-361d6e763bab None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:54.655106 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfb1b4f-6ff5-4b68-953b-361d6e763bab None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:54.655521 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfb1b4f-6ff5-4b68-953b-361d6e763bab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:54.687260 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-0bfb1b4f-6ff5-4b68-953b-361d6e763bab None None] MFA Rules not processed for user `ff4736f3d50746b0bf5d6a917b2ca0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:31:54.688563 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-0bfb1b4f-6ff5-4b68-953b-361d6e763bab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:31:54.759047 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1049/2077] 10.222.0.18 () {58 vars in 1009 bytes} [Wed Nov 2 07:31:54 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:31:54.772017 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e44011-d915-47dc-a92b-cb4b1610588e None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:31:54.773475 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-50e44011-d915-47dc-a92b-cb4b1610588e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:31:54.853488 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e44011-d915-47dc-a92b-cb4b1610588e None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:31:54.861611 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3045d383-65af-44e4-b57e-d9777f230d13 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9116f47889f4db39e0203d6cdae1593', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:31:54.862299 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3045d383-65af-44e4-b57e-d9777f230d13 service swift] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:54.862608 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3045d383-65af-44e4-b57e-d9777f230d13 service swift] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:54.862881 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3045d383-65af-44e4-b57e-d9777f230d13 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:54.872014 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3045d383-65af-44e4-b57e-d9777f230d13 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:31:54.874958 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3045d383-65af-44e4-b57e-d9777f230d13 service swift] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:31:54.884226 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1029/2078] 10.222.0.18 () {60 vars in 1384 bytes} [Wed Nov 2 07:31:54 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Nov 02 07:32:04.751461 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=57 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:32:04.880231 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=93 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:32:17.319946 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d20d20a-3cbc-4d2e-acf4-b0969215d01b None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:32:17.320862 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d20d20a-3cbc-4d2e-acf4-b0969215d01b None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:32:17.321139 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d20d20a-3cbc-4d2e-acf4-b0969215d01b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:32:17.340357 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-0d20d20a-3cbc-4d2e-acf4-b0969215d01b None None] MFA Rules not processed for user `47965dc7f5964bbfa4efff0822530965`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:32:17.341540 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-0d20d20a-3cbc-4d2e-acf4-b0969215d01b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:32:17.406724 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1050/2079] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:32:17 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:32:21.068961 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a033095a-2f31-4947-b53c-819f3160de4c None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:32:21.069472 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a033095a-2f31-4947-b53c-819f3160de4c None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:32:21.069574 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a033095a-2f31-4947-b53c-819f3160de4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:32:21.090265 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-a033095a-2f31-4947-b53c-819f3160de4c None None] MFA Rules not processed for user `ff4736f3d50746b0bf5d6a917b2ca0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:32:21.091432 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-a033095a-2f31-4947-b53c-819f3160de4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:32:21.100288 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1030/2080] 10.222.0.18 () {58 vars in 967 bytes} [Wed Nov 2 07:32:21 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:32:21.125215 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d20d20a-3cbc-4d2e-acf4-b0969215d01b None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:32:21.136922 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d20d20a-3cbc-4d2e-acf4-b0969215d01b None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:32:21.143217 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11480749-df97-4365-adce-4d0d7d65ca71 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9116f47889f4db39e0203d6cdae1593', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:32:21.143828 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11480749-df97-4365-adce-4d0d7d65ca71 service swift] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:32:21.144108 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11480749-df97-4365-adce-4d0d7d65ca71 service swift] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:32:21.144381 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11480749-df97-4365-adce-4d0d7d65ca71 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:32:21.155594 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11480749-df97-4365-adce-4d0d7d65ca71 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:32:21.158440 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11480749-df97-4365-adce-4d0d7d65ca71 service swift] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:32:21.175214 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1051/2081] 10.222.0.18 () {60 vars in 1384 bytes} [Wed Nov 2 07:32:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Nov 02 07:32:28.294870 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e201816f-5356-434c-9e77-b5e163b3439b None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:32:28.295359 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e201816f-5356-434c-9e77-b5e163b3439b None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:32:28.295465 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e201816f-5356-434c-9e77-b5e163b3439b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:32:28.314678 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-e201816f-5356-434c-9e77-b5e163b3439b None None] MFA Rules not processed for user `ff4736f3d50746b0bf5d6a917b2ca0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:32:28.315880 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-e201816f-5356-434c-9e77-b5e163b3439b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:32:28.322914 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1031/2082] 10.222.0.18 () {58 vars in 1009 bytes} [Wed Nov 2 07:32:28 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:32:28.335544 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11480749-df97-4365-adce-4d0d7d65ca71 service swift] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:32:28.346736 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11480749-df97-4365-adce-4d0d7d65ca71 service swift] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:32:28.351694 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e1ec67-3aa2-47e2-a4d3-bb1529dfc733 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9116f47889f4db39e0203d6cdae1593', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:32:28.352313 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e1ec67-3aa2-47e2-a4d3-bb1529dfc733 service swift] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:32:28.352593 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e1ec67-3aa2-47e2-a4d3-bb1529dfc733 service swift] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:32:28.352863 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e1ec67-3aa2-47e2-a4d3-bb1529dfc733 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:32:28.360709 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e1ec67-3aa2-47e2-a4d3-bb1529dfc733 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:32:28.362265 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e1ec67-3aa2-47e2-a4d3-bb1529dfc733 service swift] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:32:28.370123 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1052/2083] 10.222.0.18 () {60 vars in 1384 bytes} [Wed Nov 2 07:32:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Nov 02 07:32:30.795716 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e201816f-5356-434c-9e77-b5e163b3439b None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:32:30.797423 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-e201816f-5356-434c-9e77-b5e163b3439b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:32:30.875818 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e201816f-5356-434c-9e77-b5e163b3439b None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:32:30.881933 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87dbd1e-5c8b-415f-9d08-d022969c04c2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2fc6a156c54d0c9d325901d0f7f6fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:32:30.882584 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87dbd1e-5c8b-415f-9d08-d022969c04c2 service placement] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:32:30.882871 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87dbd1e-5c8b-415f-9d08-d022969c04c2 service placement] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:32:30.883150 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87dbd1e-5c8b-415f-9d08-d022969c04c2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:32:30.885096 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-a87dbd1e-5c8b-415f-9d08-d022969c04c2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:32:30.902492 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87dbd1e-5c8b-415f-9d08-d022969c04c2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:32:30.904874 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87dbd1e-5c8b-415f-9d08-d022969c04c2 service placement] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:32:30.922738 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1032/2084] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:32:30 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:32:34.634519 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e1ec67-3aa2-47e2-a4d3-bb1529dfc733 service swift] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:32:34.647914 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e1ec67-3aa2-47e2-a4d3-bb1529dfc733 service swift] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:32:34.651609 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5e926e-8607-4bc0-b3b9-2886778174b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3ec6ecb85f94ef59762877d9ea33ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:32:34.652744 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5e926e-8607-4bc0-b3b9-2886778174b5 service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:32:34.653181 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5e926e-8607-4bc0-b3b9-2886778174b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:32:34.653457 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5e926e-8607-4bc0-b3b9-2886778174b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:32:34.675999 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5e926e-8607-4bc0-b3b9-2886778174b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:32:34.679225 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5e926e-8607-4bc0-b3b9-2886778174b5 service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:32:34.704768 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1053/2085] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:32:34 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:32:40.918631 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=38 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:32:44.690540 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=43 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:32:45.584274 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a452e4de-9aad-486c-bcce-9be399cc5f31 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:32:45.584721 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a452e4de-9aad-486c-bcce-9be399cc5f31 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:32:45.584855 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a452e4de-9aad-486c-bcce-9be399cc5f31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:32:45.603719 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-a452e4de-9aad-486c-bcce-9be399cc5f31 None None] MFA Rules not processed for user `ff4736f3d50746b0bf5d6a917b2ca0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:32:45.604897 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-a452e4de-9aad-486c-bcce-9be399cc5f31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:32:45.612560 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1033/2086] 10.222.0.18 () {58 vars in 967 bytes} [Wed Nov 2 07:32:45 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:32:45.625394 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5e926e-8607-4bc0-b3b9-2886778174b5 service neutron] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:32:45.636532 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5e926e-8607-4bc0-b3b9-2886778174b5 service neutron] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:32:45.641707 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f501fb3-ae9b-4899-8716-51b9254b6fc0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9116f47889f4db39e0203d6cdae1593', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:32:45.642306 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f501fb3-ae9b-4899-8716-51b9254b6fc0 service swift] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:32:45.642626 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f501fb3-ae9b-4899-8716-51b9254b6fc0 service swift] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:32:45.642895 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f501fb3-ae9b-4899-8716-51b9254b6fc0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:32:45.651284 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f501fb3-ae9b-4899-8716-51b9254b6fc0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:32:45.652841 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f501fb3-ae9b-4899-8716-51b9254b6fc0 service swift] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:32:45.660853 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1054/2087] 10.222.0.18 () {60 vars in 1384 bytes} [Wed Nov 2 07:32:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Nov 02 07:32:47.572274 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a452e4de-9aad-486c-bcce-9be399cc5f31 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:32:47.573698 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-a452e4de-9aad-486c-bcce-9be399cc5f31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:32:47.638917 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a452e4de-9aad-486c-bcce-9be399cc5f31 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:32:47.644127 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92e7a26-081c-45bc-b2fb-60900d5aafbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:32:47.644758 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92e7a26-081c-45bc-b2fb-60900d5aafbc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:32:47.645040 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92e7a26-081c-45bc-b2fb-60900d5aafbc admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:32:47.645309 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92e7a26-081c-45bc-b2fb-60900d5aafbc admin admin] PATH_INFO: `/v3/users/0b55941561934c818a2b5eb2c9a19d22` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:32:47.646465 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92e7a26-081c-45bc-b2fb-60900d5aafbc admin admin] RBAC: Authorizing `identity:delete_user(user_id=0b55941561934c818a2b5eb2c9a19d22)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:32:47.648375 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92e7a26-081c-45bc-b2fb-60900d5aafbc admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:32:47.693406 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-d92e7a26-081c-45bc-b2fb-60900d5aafbc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0b55941561934c818a2b5eb2c9a19d22'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:32:47.700531 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-d92e7a26-081c-45bc-b2fb-60900d5aafbc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0b55941561934c818a2b5eb2c9a19d22'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:32:47.701138 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-d92e7a26-081c-45bc-b2fb-60900d5aafbc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0b55941561934c818a2b5eb2c9a19d22'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:32:47.708611 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-d92e7a26-081c-45bc-b2fb-60900d5aafbc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0b55941561934c818a2b5eb2c9a19d22'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:32:47.720732 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-d92e7a26-081c-45bc-b2fb-60900d5aafbc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0b55941561934c818a2b5eb2c9a19d22'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:32:47.726524 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1034/2088] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Nov 2 07:32:47 2022] DELETE /identity/v3/users/0b55941561934c818a2b5eb2c9a19d22 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:32:47.897254 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f501fb3-ae9b-4899-8716-51b9254b6fc0 service swift] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:32:47.909086 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f501fb3-ae9b-4899-8716-51b9254b6fc0 service swift] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:32:47.970773 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d03ed8-f4a6-414f-88a5-1b6a98c99942 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:32:47.971394 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d03ed8-f4a6-414f-88a5-1b6a98c99942 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:32:47.971671 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d03ed8-f4a6-414f-88a5-1b6a98c99942 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:32:47.971940 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d03ed8-f4a6-414f-88a5-1b6a98c99942 admin admin] PATH_INFO: `/v3/projects/155356fb94b74eaba3a8465d1a1f21ff` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:32:47.973037 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d03ed8-f4a6-414f-88a5-1b6a98c99942 admin admin] RBAC: Authorizing `identity:delete_project(project_id=155356fb94b74eaba3a8465d1a1f21ff)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:32:47.974956 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d03ed8-f4a6-414f-88a5-1b6a98c99942 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:32:48.020352 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-86d03ed8-f4a6-414f-88a5-1b6a98c99942 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '155356fb94b74eaba3a8465d1a1f21ff', 'request_id': 'req-86d03ed8-f4a6-414f-88a5-1b6a98c99942'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:32:48.026442 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-86d03ed8-f4a6-414f-88a5-1b6a98c99942 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '155356fb94b74eaba3a8465d1a1f21ff', 'request_id': 'req-86d03ed8-f4a6-414f-88a5-1b6a98c99942'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:32:48.032129 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-86d03ed8-f4a6-414f-88a5-1b6a98c99942 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '155356fb94b74eaba3a8465d1a1f21ff', 'request_id': 'req-86d03ed8-f4a6-414f-88a5-1b6a98c99942'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:32:48.045132 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-86d03ed8-f4a6-414f-88a5-1b6a98c99942 admin admin] The token cache is being invalidate because project 155356fb94b74eaba3a8465d1a1f21ff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:32:48.045419 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-86d03ed8-f4a6-414f-88a5-1b6a98c99942 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:32:48.046074 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-86d03ed8-f4a6-414f-88a5-1b6a98c99942 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:32:48.047702 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1055/2089] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Nov 2 07:32:47 2022] DELETE /identity/v3/projects/155356fb94b74eaba3a8465d1a1f21ff => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:32:48.052543 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02874279-ee83-45ff-a1c2-ae949e6592fb None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:32:48.052834 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02874279-ee83-45ff-a1c2-ae949e6592fb None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:32:48.053112 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02874279-ee83-45ff-a1c2-ae949e6592fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:32:48.072818 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-02874279-ee83-45ff-a1c2-ae949e6592fb None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:32:48.073940 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-02874279-ee83-45ff-a1c2-ae949e6592fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:32:48.131828 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1035/2090] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:32:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:32:48.140701 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c222864-7d61-4482-b5d1-83dec316f78a None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:32:48.141050 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c222864-7d61-4482-b5d1-83dec316f78a None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:32:48.141323 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c222864-7d61-4482-b5d1-83dec316f78a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:32:48.161055 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-6c222864-7d61-4482-b5d1-83dec316f78a None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:32:48.162230 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-6c222864-7d61-4482-b5d1-83dec316f78a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:32:48.169867 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1056/2091] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:32:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:32:48.174347 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02874279-ee83-45ff-a1c2-ae949e6592fb None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:32:48.183832 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02874279-ee83-45ff-a1c2-ae949e6592fb None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:32:48.189008 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3100dab-0110-4cd1-9818-905706045ef3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:32:48.189599 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3100dab-0110-4cd1-9818-905706045ef3 admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:32:48.189892 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3100dab-0110-4cd1-9818-905706045ef3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:32:48.190145 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3100dab-0110-4cd1-9818-905706045ef3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:32:48.190824 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3100dab-0110-4cd1-9818-905706045ef3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:32:48.191126 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3100dab-0110-4cd1-9818-905706045ef3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:32:48.192739 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3100dab-0110-4cd1-9818-905706045ef3 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:32:48.201133 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:32:48.205380 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1036/2092] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Nov 2 07:32:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:32:48.209703 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c222864-7d61-4482-b5d1-83dec316f78a None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:32:48.218751 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c222864-7d61-4482-b5d1-83dec316f78a None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:32:48.223363 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3de13b-2e67-4744-88d8-fb867e7fb3e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:32:48.223975 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3de13b-2e67-4744-88d8-fb867e7fb3e8 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:32:48.224251 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3de13b-2e67-4744-88d8-fb867e7fb3e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:32:48.224518 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3de13b-2e67-4744-88d8-fb867e7fb3e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:32:48.225131 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3de13b-2e67-4744-88d8-fb867e7fb3e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:32:48.227067 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3de13b-2e67-4744-88d8-fb867e7fb3e8 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:32:48.254507 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1057/2093] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Nov 2 07:32:48 2022] POST /identity/v3/projects => generated 393 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:32:48.258584 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3100dab-0110-4cd1-9818-905706045ef3 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:32:48.268942 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3100dab-0110-4cd1-9818-905706045ef3 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:32:48.323662 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ecaaf7-4915-4822-841b-32d78f378850 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:32:48.324300 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ecaaf7-4915-4822-841b-32d78f378850 admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:32:48.324609 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ecaaf7-4915-4822-841b-32d78f378850 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:32:48.324901 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ecaaf7-4915-4822-841b-32d78f378850 admin admin] PATH_INFO: `/v3/users` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:32:48.325550 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ecaaf7-4915-4822-841b-32d78f378850 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:32:48.327554 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ecaaf7-4915-4822-841b-32d78f378850 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:32:48.349945 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1037/2094] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Nov 2 07:32:48 2022] POST /identity/v3/users => generated 297 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 07:32:48.354183 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3de13b-2e67-4744-88d8-fb867e7fb3e8 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:32:48.364856 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3de13b-2e67-4744-88d8-fb867e7fb3e8 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:32:48.369372 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecbd141-918d-4b5a-9570-b0b387517396 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:32:48.369954 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecbd141-918d-4b5a-9570-b0b387517396 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:32:48.370229 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecbd141-918d-4b5a-9570-b0b387517396 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:32:48.370535 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecbd141-918d-4b5a-9570-b0b387517396 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:32:48.371163 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eecbd141-918d-4b5a-9570-b0b387517396 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:32:48.372988 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eecbd141-918d-4b5a-9570-b0b387517396 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:32:48.383225 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1058/2095] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:32:48 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:32:48.387084 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ecaaf7-4915-4822-841b-32d78f378850 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:32:48.397179 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ecaaf7-4915-4822-841b-32d78f378850 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:32:48.401738 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662afe93-faa8-4331-8576-097148b31651 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:32:48.402350 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662afe93-faa8-4331-8576-097148b31651 admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:32:48.402652 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662afe93-faa8-4331-8576-097148b31651 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:32:48.402921 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662afe93-faa8-4331-8576-097148b31651 admin admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:32:48.403549 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662afe93-faa8-4331-8576-097148b31651 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:32:48.405316 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662afe93-faa8-4331-8576-097148b31651 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:32:48.416148 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1038/2096] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:32:48 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:32:48.420196 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecbd141-918d-4b5a-9570-b0b387517396 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:32:48.431178 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecbd141-918d-4b5a-9570-b0b387517396 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:32:48.435806 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23ea717-ee65-4b30-b075-3ca7f52250fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:32:48.436445 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23ea717-ee65-4b30-b075-3ca7f52250fd admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:32:48.436726 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23ea717-ee65-4b30-b075-3ca7f52250fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:32:48.437002 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23ea717-ee65-4b30-b075-3ca7f52250fd admin admin] PATH_INFO: `/v3/projects/95fb3f8f6f73418f8b751917e23e6f0f/users/5ed52fc7a91d4961b5962ac184eb62e6/roles/b494fedd00f24f0b84a529e8672a156a` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:32:48.453453 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23ea717-ee65-4b30-b075-3ca7f52250fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=95fb3f8f6f73418f8b751917e23e6f0f, user_id=5ed52fc7a91d4961b5962ac184eb62e6, role_id=b494fedd00f24f0b84a529e8672a156a)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:32:48.456175 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23ea717-ee65-4b30-b075-3ca7f52250fd admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:32:48.469929 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1059/2097] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Nov 2 07:32:48 2022] PUT /identity/v3/projects/95fb3f8f6f73418f8b751917e23e6f0f/users/5ed52fc7a91d4961b5962ac184eb62e6/roles/b494fedd00f24f0b84a529e8672a156a => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:32:48.480354 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53335d9-fe90-4a66-b7c4-d2abb20ebe7b None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:32:48.480635 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53335d9-fe90-4a66-b7c4-d2abb20ebe7b None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:32:48.480926 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53335d9-fe90-4a66-b7c4-d2abb20ebe7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:32:48.512010 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-a53335d9-fe90-4a66-b7c4-d2abb20ebe7b None None] MFA Rules not processed for user `5ed52fc7a91d4961b5962ac184eb62e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:32:48.513603 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-a53335d9-fe90-4a66-b7c4-d2abb20ebe7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:32:48.582554 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1039/2098] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:32:48 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:32:48.594437 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23ea717-ee65-4b30-b075-3ca7f52250fd admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:32:48.596155 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-b23ea717-ee65-4b30-b075-3ca7f52250fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:32:48.611493 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:32:48.662085 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23ea717-ee65-4b30-b075-3ca7f52250fd admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:32:48.665687 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf98346c-cd25-48fb-a1f3-b6ca7d8d9919 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43bb326fb82e4d2c83fb2eaf91860dc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:32:48.666291 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf98346c-cd25-48fb-a1f3-b6ca7d8d9919 service cinder] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:32:48.666623 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf98346c-cd25-48fb-a1f3-b6ca7d8d9919 service cinder] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:32:48.666874 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf98346c-cd25-48fb-a1f3-b6ca7d8d9919 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:32:48.675807 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf98346c-cd25-48fb-a1f3-b6ca7d8d9919 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:32:48.677234 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf98346c-cd25-48fb-a1f3-b6ca7d8d9919 service cinder] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:32:48.686207 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1060/2099] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:32:48 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:32:56.755125 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53335d9-fe90-4a66-b7c4-d2abb20ebe7b None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:32:56.817787 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53335d9-fe90-4a66-b7c4-d2abb20ebe7b None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:32:56.822273 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a491a1c-08fe-412c-a625-fcd299d34c70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:32:56.822924 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a491a1c-08fe-412c-a625-fcd299d34c70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:32:56.823204 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a491a1c-08fe-412c-a625-fcd299d34c70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:32:56.823477 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a491a1c-08fe-412c-a625-fcd299d34c70 admin admin] PATH_INFO: `/v3/users/5ed52fc7a91d4961b5962ac184eb62e6` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:32:56.824578 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a491a1c-08fe-412c-a625-fcd299d34c70 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5ed52fc7a91d4961b5962ac184eb62e6)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:32:56.826539 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a491a1c-08fe-412c-a625-fcd299d34c70 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:32:56.837047 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:32:56.872601 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-6a491a1c-08fe-412c-a625-fcd299d34c70 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5ed52fc7a91d4961b5962ac184eb62e6'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:32:56.880552 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-6a491a1c-08fe-412c-a625-fcd299d34c70 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5ed52fc7a91d4961b5962ac184eb62e6'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:32:56.881159 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-6a491a1c-08fe-412c-a625-fcd299d34c70 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5ed52fc7a91d4961b5962ac184eb62e6'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:32:56.887991 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-6a491a1c-08fe-412c-a625-fcd299d34c70 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5ed52fc7a91d4961b5962ac184eb62e6'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:32:56.899607 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-6a491a1c-08fe-412c-a625-fcd299d34c70 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5ed52fc7a91d4961b5962ac184eb62e6'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:32:56.905378 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1040/2100] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Nov 2 07:32:56 2022] DELETE /identity/v3/users/5ed52fc7a91d4961b5962ac184eb62e6 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:32:56.910640 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f06e4e7-d5b4-4874-8ad6-098362574689 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:32:56.911140 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f06e4e7-d5b4-4874-8ad6-098362574689 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:32:56.911244 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f06e4e7-d5b4-4874-8ad6-098362574689 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:32:56.930992 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-2f06e4e7-d5b4-4874-8ad6-098362574689 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:32:56.932174 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-2f06e4e7-d5b4-4874-8ad6-098362574689 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:32:56.989387 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1061/2101] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:32:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:32:56.999193 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a491a1c-08fe-412c-a625-fcd299d34c70 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:32:57.000605 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-6a491a1c-08fe-412c-a625-fcd299d34c70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:32:57.056340 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a491a1c-08fe-412c-a625-fcd299d34c70 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:32:57.059931 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0558686c-c990-49ee-a5ff-80e4b4fc2ee3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3ec6ecb85f94ef59762877d9ea33ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:32:57.060525 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0558686c-c990-49ee-a5ff-80e4b4fc2ee3 service neutron] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:32:57.060805 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0558686c-c990-49ee-a5ff-80e4b4fc2ee3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:32:57.061072 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0558686c-c990-49ee-a5ff-80e4b4fc2ee3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:32:57.070254 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0558686c-c990-49ee-a5ff-80e4b4fc2ee3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:32:57.071703 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0558686c-c990-49ee-a5ff-80e4b4fc2ee3 service neutron] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:32:57.081082 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1041/2102] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:32:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:32:57.346080 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f06e4e7-d5b4-4874-8ad6-098362574689 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:32:57.347514 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-2f06e4e7-d5b4-4874-8ad6-098362574689 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:32:57.359491 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f06e4e7-d5b4-4874-8ad6-098362574689 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:32:57.364151 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3a0a03-66d0-4e88-8022-c6b3f292c5cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:32:57.364764 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3a0a03-66d0-4e88-8022-c6b3f292c5cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:32:57.365041 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3a0a03-66d0-4e88-8022-c6b3f292c5cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:32:57.365308 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3a0a03-66d0-4e88-8022-c6b3f292c5cc admin admin] PATH_INFO: `/v3/projects/95fb3f8f6f73418f8b751917e23e6f0f` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:32:57.366494 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3a0a03-66d0-4e88-8022-c6b3f292c5cc admin admin] RBAC: Authorizing `identity:delete_project(project_id=95fb3f8f6f73418f8b751917e23e6f0f)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:32:57.368433 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3a0a03-66d0-4e88-8022-c6b3f292c5cc admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:32:57.416045 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-4f3a0a03-66d0-4e88-8022-c6b3f292c5cc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '95fb3f8f6f73418f8b751917e23e6f0f', 'request_id': 'req-4f3a0a03-66d0-4e88-8022-c6b3f292c5cc'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:32:57.422070 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-4f3a0a03-66d0-4e88-8022-c6b3f292c5cc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '95fb3f8f6f73418f8b751917e23e6f0f', 'request_id': 'req-4f3a0a03-66d0-4e88-8022-c6b3f292c5cc'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:32:57.427613 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-4f3a0a03-66d0-4e88-8022-c6b3f292c5cc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '95fb3f8f6f73418f8b751917e23e6f0f', 'request_id': 'req-4f3a0a03-66d0-4e88-8022-c6b3f292c5cc'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:32:57.441833 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-4f3a0a03-66d0-4e88-8022-c6b3f292c5cc admin admin] The token cache is being invalidate because project 95fb3f8f6f73418f8b751917e23e6f0f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:32:57.442152 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-4f3a0a03-66d0-4e88-8022-c6b3f292c5cc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:32:57.443033 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-4f3a0a03-66d0-4e88-8022-c6b3f292c5cc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:32:57.444621 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1062/2103] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Nov 2 07:32:57 2022] DELETE /identity/v3/projects/95fb3f8f6f73418f8b751917e23e6f0f => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 07:33:07.076725 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:DELETE=10,keystone:SELECT=48,keystone:INSERT=1 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:33:07.437607 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=74,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:33:19.072505 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737f30af-1847-4fcb-bcf4-f554f0d246c4 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:33:19.072980 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737f30af-1847-4fcb-bcf4-f554f0d246c4 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:33:19.073088 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737f30af-1847-4fcb-bcf4-f554f0d246c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:33:19.092354 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-737f30af-1847-4fcb-bcf4-f554f0d246c4 None None] MFA Rules not processed for user `2455ebfc92ba4d398a6279d6102a0408`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:33:19.093543 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-737f30af-1847-4fcb-bcf4-f554f0d246c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:33:19.150802 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1042/2104] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:33:19 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:33:21.369815 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27f75bf-d1a2-4f3a-afd2-c8ade54da7c2 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:33:21.370304 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27f75bf-d1a2-4f3a-afd2-c8ade54da7c2 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:33:21.370521 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27f75bf-d1a2-4f3a-afd2-c8ade54da7c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:33:21.389754 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-c27f75bf-d1a2-4f3a-afd2-c8ade54da7c2 None None] MFA Rules not processed for user `0e571b0f4d6f4fafb71939f501b8deec`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:33:21.390984 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-c27f75bf-d1a2-4f3a-afd2-c8ade54da7c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:33:21.453811 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1063/2105] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:33:21 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:33:29.144400 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=25 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:33:31.446192 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=25 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:33:38.553734 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c33d2b-5296-4081-a793-be4a837a18b9 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:33:38.554318 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c33d2b-5296-4081-a793-be4a837a18b9 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:33:38.554471 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c33d2b-5296-4081-a793-be4a837a18b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:33:38.572826 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-e4c33d2b-5296-4081-a793-be4a837a18b9 None None] MFA Rules not processed for user `47965dc7f5964bbfa4efff0822530965`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:33:38.574007 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-e4c33d2b-5296-4081-a793-be4a837a18b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:33:38.627316 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1043/2106] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:33:38 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:33:48.620504 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=25 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:33:56.227429 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06207513-c86f-45c7-aac5-174c78be0d8a None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:33:56.227901 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06207513-c86f-45c7-aac5-174c78be0d8a None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:33:56.228008 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06207513-c86f-45c7-aac5-174c78be0d8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:33:56.246493 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-06207513-c86f-45c7-aac5-174c78be0d8a None None] MFA Rules not processed for user `fe26aea6bb9d4dcebf046da719b2c244`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:33:56.247679 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-06207513-c86f-45c7-aac5-174c78be0d8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:33:56.301426 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1064/2107] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:33:56 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:34:06.294694 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=25 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:34:13.513648 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c33d2b-5296-4081-a793-be4a837a18b9 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:34:13.515081 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-e4c33d2b-5296-4081-a793-be4a837a18b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:34:13.585083 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c33d2b-5296-4081-a793-be4a837a18b9 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:34:13.589426 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f18d66-4ef3-4d6d-aa17-61c7e766203f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5353f2af0f44d668e2a269647aa1dc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:34:13.590043 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f18d66-4ef3-4d6d-aa17-61c7e766203f service glance] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:34:13.590326 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f18d66-4ef3-4d6d-aa17-61c7e766203f service glance] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:34:13.590746 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f18d66-4ef3-4d6d-aa17-61c7e766203f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:34:13.592384 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-47f18d66-4ef3-4d6d-aa17-61c7e766203f service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:34:13.681116 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f18d66-4ef3-4d6d-aa17-61c7e766203f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:34:13.682589 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f18d66-4ef3-4d6d-aa17-61c7e766203f service glance] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:34:13.696915 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1044/2108] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:34:13 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 07:34:15.546481 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06207513-c86f-45c7-aac5-174c78be0d8a None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:34:15.547898 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-06207513-c86f-45c7-aac5-174c78be0d8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:34:15.607933 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06207513-c86f-45c7-aac5-174c78be0d8a None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:34:15.611582 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6085414b-e2f7-499e-b097-eb579558bd48 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3ec6ecb85f94ef59762877d9ea33ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:34:15.612185 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6085414b-e2f7-499e-b097-eb579558bd48 service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:34:15.612469 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6085414b-e2f7-499e-b097-eb579558bd48 service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:34:15.612739 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6085414b-e2f7-499e-b097-eb579558bd48 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:34:15.614587 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-6085414b-e2f7-499e-b097-eb579558bd48 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:34:15.628894 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6085414b-e2f7-499e-b097-eb579558bd48 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:34:15.630881 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6085414b-e2f7-499e-b097-eb579558bd48 service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:34:15.642896 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1065/2109] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:34:15 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:34:21.679503 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f18d66-4ef3-4d6d-aa17-61c7e766203f service glance] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:34:21.680873 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-47f18d66-4ef3-4d6d-aa17-61c7e766203f service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:34:21.691887 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f18d66-4ef3-4d6d-aa17-61c7e766203f service glance] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:34:21.695500 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71595c2d-9fb4-42c8-8c05-5936ebb5baee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3ec6ecb85f94ef59762877d9ea33ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:34:21.696084 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71595c2d-9fb4-42c8-8c05-5936ebb5baee service neutron] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:34:21.696357 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71595c2d-9fb4-42c8-8c05-5936ebb5baee service neutron] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:34:21.696629 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71595c2d-9fb4-42c8-8c05-5936ebb5baee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:34:21.698245 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-71595c2d-9fb4-42c8-8c05-5936ebb5baee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:34:21.759300 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71595c2d-9fb4-42c8-8c05-5936ebb5baee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:34:21.760711 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71595c2d-9fb4-42c8-8c05-5936ebb5baee service neutron] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:34:21.770252 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1045/2110] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:34:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:34:25.477764 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6085414b-e2f7-499e-b097-eb579558bd48 service neutron] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:34:25.479230 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-6085414b-e2f7-499e-b097-eb579558bd48 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:34:25.494050 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6085414b-e2f7-499e-b097-eb579558bd48 service neutron] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:34:25.498490 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57578fe9-d4f8-4ddb-a671-d972c012fa87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:34:25.499104 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57578fe9-d4f8-4ddb-a671-d972c012fa87 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:34:25.499378 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57578fe9-d4f8-4ddb-a671-d972c012fa87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:34:25.499648 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57578fe9-d4f8-4ddb-a671-d972c012fa87 admin admin] PATH_INFO: `/v3/users/0e571b0f4d6f4fafb71939f501b8deec` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:34:25.500739 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57578fe9-d4f8-4ddb-a671-d972c012fa87 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0e571b0f4d6f4fafb71939f501b8deec)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:34:25.502914 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57578fe9-d4f8-4ddb-a671-d972c012fa87 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:34:25.549817 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-57578fe9-d4f8-4ddb-a671-d972c012fa87 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0e571b0f4d6f4fafb71939f501b8deec'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:34:25.557051 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-57578fe9-d4f8-4ddb-a671-d972c012fa87 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0e571b0f4d6f4fafb71939f501b8deec'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:34:25.557657 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-57578fe9-d4f8-4ddb-a671-d972c012fa87 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0e571b0f4d6f4fafb71939f501b8deec'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:34:25.564447 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-57578fe9-d4f8-4ddb-a671-d972c012fa87 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0e571b0f4d6f4fafb71939f501b8deec'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:34:25.565079 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-57578fe9-d4f8-4ddb-a671-d972c012fa87 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0e571b0f4d6f4fafb71939f501b8deec'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:34:25.582339 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1066/2111] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Nov 2 07:34:25 2022] DELETE /identity/v3/users/0e571b0f4d6f4fafb71939f501b8deec => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:34:25.769076 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71595c2d-9fb4-42c8-8c05-5936ebb5baee service neutron] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:34:25.779695 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71595c2d-9fb4-42c8-8c05-5936ebb5baee service neutron] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:34:25.815869 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:34:25.843045 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d391dc0-8d39-4218-ac8c-856f9f018a81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:34:25.843681 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d391dc0-8d39-4218-ac8c-856f9f018a81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:34:25.843962 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d391dc0-8d39-4218-ac8c-856f9f018a81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:34:25.844237 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d391dc0-8d39-4218-ac8c-856f9f018a81 admin admin] PATH_INFO: `/v3/projects/c60cb95cbf734d9b852af2bdcad4e1c0` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:34:25.845347 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d391dc0-8d39-4218-ac8c-856f9f018a81 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c60cb95cbf734d9b852af2bdcad4e1c0)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:34:25.847376 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d391dc0-8d39-4218-ac8c-856f9f018a81 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:34:25.896010 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-0d391dc0-8d39-4218-ac8c-856f9f018a81 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c60cb95cbf734d9b852af2bdcad4e1c0', 'request_id': 'req-0d391dc0-8d39-4218-ac8c-856f9f018a81'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:34:25.902048 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-0d391dc0-8d39-4218-ac8c-856f9f018a81 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c60cb95cbf734d9b852af2bdcad4e1c0', 'request_id': 'req-0d391dc0-8d39-4218-ac8c-856f9f018a81'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:34:25.910398 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-0d391dc0-8d39-4218-ac8c-856f9f018a81 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c60cb95cbf734d9b852af2bdcad4e1c0', 'request_id': 'req-0d391dc0-8d39-4218-ac8c-856f9f018a81'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:34:25.923728 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-0d391dc0-8d39-4218-ac8c-856f9f018a81 admin admin] The token cache is being invalidate because project c60cb95cbf734d9b852af2bdcad4e1c0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81501) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:34:25.924025 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-0d391dc0-8d39-4218-ac8c-856f9f018a81 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:34:25.924670 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-0d391dc0-8d39-4218-ac8c-856f9f018a81 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:34:25.930805 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1046/2112] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Nov 2 07:34:25 2022] DELETE /identity/v3/projects/c60cb95cbf734d9b852af2bdcad4e1c0 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:34:25.932481 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972dd371-7182-4140-8398-4f2be6791cf1 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:34:25.932749 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972dd371-7182-4140-8398-4f2be6791cf1 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:34:25.932947 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972dd371-7182-4140-8398-4f2be6791cf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:34:25.958022 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-972dd371-7182-4140-8398-4f2be6791cf1 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:34:25.959397 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-972dd371-7182-4140-8398-4f2be6791cf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:34:26.024251 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1067/2113] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:34:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:34:26.032341 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d916ed-1906-4bd1-927a-7abf03cee19d None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:34:26.032628 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d916ed-1906-4bd1-927a-7abf03cee19d None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:34:26.032902 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d916ed-1906-4bd1-927a-7abf03cee19d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:34:26.053751 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-76d916ed-1906-4bd1-927a-7abf03cee19d None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:34:26.055139 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-76d916ed-1906-4bd1-927a-7abf03cee19d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:34:26.063122 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1047/2114] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:34:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:34:26.067323 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972dd371-7182-4140-8398-4f2be6791cf1 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:34:26.077056 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972dd371-7182-4140-8398-4f2be6791cf1 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:34:26.081832 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88a3475-a47a-43fc-9621-314dc37984e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:34:26.082577 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88a3475-a47a-43fc-9621-314dc37984e3 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:34:26.082931 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88a3475-a47a-43fc-9621-314dc37984e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:34:26.083275 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88a3475-a47a-43fc-9621-314dc37984e3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:34:26.084021 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88a3475-a47a-43fc-9621-314dc37984e3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:34:26.084392 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88a3475-a47a-43fc-9621-314dc37984e3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:34:26.086150 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88a3475-a47a-43fc-9621-314dc37984e3 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:34:26.098688 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1068/2115] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Nov 2 07:34:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:34:26.103424 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d916ed-1906-4bd1-927a-7abf03cee19d None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:34:26.112761 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d916ed-1906-4bd1-927a-7abf03cee19d None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:34:26.118068 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7423297e-94fa-443a-9b22-6decf6cdac7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:34:26.118707 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7423297e-94fa-443a-9b22-6decf6cdac7d admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:34:26.118984 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7423297e-94fa-443a-9b22-6decf6cdac7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:34:26.119253 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7423297e-94fa-443a-9b22-6decf6cdac7d admin admin] PATH_INFO: `/v3/projects` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:34:26.119946 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7423297e-94fa-443a-9b22-6decf6cdac7d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:34:26.121825 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7423297e-94fa-443a-9b22-6decf6cdac7d admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:34:26.154305 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1048/2116] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Nov 2 07:34:26 2022] POST /identity/v3/projects => generated 365 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:34:26.155496 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88a3475-a47a-43fc-9621-314dc37984e3 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:34:26.160969 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:34:26.168383 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88a3475-a47a-43fc-9621-314dc37984e3 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:34:26.231060 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc98902a-c0c6-4610-9cfb-d721735996f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:34:26.231664 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc98902a-c0c6-4610-9cfb-d721735996f7 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:34:26.231941 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc98902a-c0c6-4610-9cfb-d721735996f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:34:26.232209 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc98902a-c0c6-4610-9cfb-d721735996f7 admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:34:26.232854 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc98902a-c0c6-4610-9cfb-d721735996f7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:34:26.234788 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc98902a-c0c6-4610-9cfb-d721735996f7 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:34:26.257154 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1069/2117] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Nov 2 07:34:26 2022] POST /identity/v3/users => generated 283 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 07:34:26.261247 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7423297e-94fa-443a-9b22-6decf6cdac7d admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:34:26.271540 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7423297e-94fa-443a-9b22-6decf6cdac7d admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:34:26.276229 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9cf9a0-cfb6-40f1-a6cd-cd4e525ba974 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:34:26.276834 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9cf9a0-cfb6-40f1-a6cd-cd4e525ba974 admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:34:26.277112 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9cf9a0-cfb6-40f1-a6cd-cd4e525ba974 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:34:26.277384 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9cf9a0-cfb6-40f1-a6cd-cd4e525ba974 admin admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:34:26.278020 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9cf9a0-cfb6-40f1-a6cd-cd4e525ba974 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:34:26.279676 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9cf9a0-cfb6-40f1-a6cd-cd4e525ba974 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:34:26.290386 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1049/2118] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:34:26 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 07:34:26.294377 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc98902a-c0c6-4610-9cfb-d721735996f7 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:34:26.305413 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc98902a-c0c6-4610-9cfb-d721735996f7 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:34:26.310271 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b032851d-dd12-477f-9697-521f0ec63191 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:34:26.310939 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b032851d-dd12-477f-9697-521f0ec63191 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:34:26.311221 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b032851d-dd12-477f-9697-521f0ec63191 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:34:26.311489 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b032851d-dd12-477f-9697-521f0ec63191 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:34:26.312125 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b032851d-dd12-477f-9697-521f0ec63191 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:34:26.313712 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b032851d-dd12-477f-9697-521f0ec63191 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:34:26.328844 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1070/2119] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:34:26 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:34:26.330090 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9cf9a0-cfb6-40f1-a6cd-cd4e525ba974 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:34:26.339874 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9cf9a0-cfb6-40f1-a6cd-cd4e525ba974 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:34:26.344468 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702666d9-e070-429b-90e9-de821dd894f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:34:26.345107 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702666d9-e070-429b-90e9-de821dd894f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:34:26.345386 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702666d9-e070-429b-90e9-de821dd894f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:34:26.345656 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702666d9-e070-429b-90e9-de821dd894f4 admin admin] PATH_INFO: `/v3/projects/14f1b596ab4142c8befcc0e0b878c0ed/users/89622fb23d2b42acb3877d4dd6817675/roles/b494fedd00f24f0b84a529e8672a156a` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:34:26.361847 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702666d9-e070-429b-90e9-de821dd894f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=14f1b596ab4142c8befcc0e0b878c0ed, user_id=89622fb23d2b42acb3877d4dd6817675, role_id=b494fedd00f24f0b84a529e8672a156a)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:34:26.364826 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702666d9-e070-429b-90e9-de821dd894f4 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:34:26.379460 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1050/2120] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Nov 2 07:34:26 2022] PUT /identity/v3/projects/14f1b596ab4142c8befcc0e0b878c0ed/users/89622fb23d2b42acb3877d4dd6817675/roles/b494fedd00f24f0b84a529e8672a156a => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:34:26.390852 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96af5eb8-e415-40c9-a7cc-d776f542ce78 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:34:26.391181 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96af5eb8-e415-40c9-a7cc-d776f542ce78 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:34:26.391595 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96af5eb8-e415-40c9-a7cc-d776f542ce78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:34:26.421724 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-96af5eb8-e415-40c9-a7cc-d776f542ce78 None None] MFA Rules not processed for user `89622fb23d2b42acb3877d4dd6817675`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:34:26.423216 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-96af5eb8-e415-40c9-a7cc-d776f542ce78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:34:26.492340 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1071/2121] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:34:26 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:34:26.496801 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702666d9-e070-429b-90e9-de821dd894f4 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:34:26.557395 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702666d9-e070-429b-90e9-de821dd894f4 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:34:26.561948 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591a7468-c36c-442d-ad08-f74557db00a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:34:26.562568 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591a7468-c36c-442d-ad08-f74557db00a3 admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:34:26.562847 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591a7468-c36c-442d-ad08-f74557db00a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:34:26.563123 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591a7468-c36c-442d-ad08-f74557db00a3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:34:26.563737 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591a7468-c36c-442d-ad08-f74557db00a3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:34:26.565623 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591a7468-c36c-442d-ad08-f74557db00a3 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:34:26.571397 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:34:26.594811 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1051/2122] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Nov 2 07:34:26 2022] POST /identity/v3/projects => generated 361 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:34:26.598881 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96af5eb8-e415-40c9-a7cc-d776f542ce78 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:34:26.609714 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96af5eb8-e415-40c9-a7cc-d776f542ce78 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:34:26.667576 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cd4bfe-6a30-4b4c-9a50-b6a09919bdf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:34:26.668173 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cd4bfe-6a30-4b4c-9a50-b6a09919bdf4 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:34:26.668449 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cd4bfe-6a30-4b4c-9a50-b6a09919bdf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:34:26.668712 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cd4bfe-6a30-4b4c-9a50-b6a09919bdf4 admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:34:26.669322 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15cd4bfe-6a30-4b4c-9a50-b6a09919bdf4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:34:26.671246 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15cd4bfe-6a30-4b4c-9a50-b6a09919bdf4 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:34:26.688392 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:34:26.693874 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1072/2123] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Nov 2 07:34:26 2022] POST /identity/v3/users => generated 287 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:34:26.697626 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591a7468-c36c-442d-ad08-f74557db00a3 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:34:26.707664 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591a7468-c36c-442d-ad08-f74557db00a3 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:34:26.713092 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd80e93-292e-4985-a35c-34c90183288c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:34:26.713684 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd80e93-292e-4985-a35c-34c90183288c admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:34:26.713963 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd80e93-292e-4985-a35c-34c90183288c admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:34:26.714232 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd80e93-292e-4985-a35c-34c90183288c admin admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:34:26.714896 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd80e93-292e-4985-a35c-34c90183288c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:34:26.716479 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd80e93-292e-4985-a35c-34c90183288c admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:34:26.727038 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1052/2124] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:34:26 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 07:34:26.731238 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cd4bfe-6a30-4b4c-9a50-b6a09919bdf4 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:34:26.741271 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cd4bfe-6a30-4b4c-9a50-b6a09919bdf4 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:34:26.746023 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a6beb5-5cdf-4f45-8342-c8c575d62461 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:34:26.746695 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a6beb5-5cdf-4f45-8342-c8c575d62461 admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:34:26.746975 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a6beb5-5cdf-4f45-8342-c8c575d62461 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:34:26.747246 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a6beb5-5cdf-4f45-8342-c8c575d62461 admin admin] PATH_INFO: `/v3/projects/b9680632ba4f4039a8a16feacfc7f861/users/f306ac5d4d7d429f93bbce942e92b1fe/roles/c4bf019fc6c44c71a6c984f6d0bab75f` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:34:26.762766 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a6beb5-5cdf-4f45-8342-c8c575d62461 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9680632ba4f4039a8a16feacfc7f861, user_id=f306ac5d4d7d429f93bbce942e92b1fe, role_id=c4bf019fc6c44c71a6c984f6d0bab75f)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:34:26.765391 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a6beb5-5cdf-4f45-8342-c8c575d62461 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:34:26.778723 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1073/2125] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Nov 2 07:34:26 2022] PUT /identity/v3/projects/b9680632ba4f4039a8a16feacfc7f861/users/f306ac5d4d7d429f93bbce942e92b1fe/roles/c4bf019fc6c44c71a6c984f6d0bab75f => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 07:34:26.782620 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd80e93-292e-4985-a35c-34c90183288c admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:34:26.793831 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd80e93-292e-4985-a35c-34c90183288c admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:34:26.851614 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0709ce-6aa6-4bb1-8eb6-581adb87d5d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:34:26.852283 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0709ce-6aa6-4bb1-8eb6-581adb87d5d8 admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:34:26.852563 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0709ce-6aa6-4bb1-8eb6-581adb87d5d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:34:26.852862 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0709ce-6aa6-4bb1-8eb6-581adb87d5d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:34:26.853534 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0709ce-6aa6-4bb1-8eb6-581adb87d5d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:34:26.855183 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0709ce-6aa6-4bb1-8eb6-581adb87d5d8 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:34:26.864717 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1053/2126] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:34:26 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 07:34:26.869505 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a6beb5-5cdf-4f45-8342-c8c575d62461 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:34:26.879850 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a6beb5-5cdf-4f45-8342-c8c575d62461 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:34:26.884678 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc4e383-75cf-4768-ac1b-439b5b9f1af6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:34:26.885328 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc4e383-75cf-4768-ac1b-439b5b9f1af6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:34:26.885607 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc4e383-75cf-4768-ac1b-439b5b9f1af6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:34:26.885878 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc4e383-75cf-4768-ac1b-439b5b9f1af6 admin admin] PATH_INFO: `/v3/projects/b9680632ba4f4039a8a16feacfc7f861/users/f306ac5d4d7d429f93bbce942e92b1fe/roles/c4bf019fc6c44c71a6c984f6d0bab75f` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:34:26.887788 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc4e383-75cf-4768-ac1b-439b5b9f1af6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9680632ba4f4039a8a16feacfc7f861, user_id=f306ac5d4d7d429f93bbce942e92b1fe, role_id=c4bf019fc6c44c71a6c984f6d0bab75f)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:34:26.890408 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc4e383-75cf-4768-ac1b-439b5b9f1af6 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:34:26.900808 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1074/2127] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Nov 2 07:34:26 2022] PUT /identity/v3/projects/b9680632ba4f4039a8a16feacfc7f861/users/f306ac5d4d7d429f93bbce942e92b1fe/roles/c4bf019fc6c44c71a6c984f6d0bab75f => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:34:26.912557 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe0ddbe-c9d1-4168-abe3-80b672c55e60 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:34:26.913063 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe0ddbe-c9d1-4168-abe3-80b672c55e60 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:34:26.913426 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe0ddbe-c9d1-4168-abe3-80b672c55e60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:34:26.942313 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-cbe0ddbe-c9d1-4168-abe3-80b672c55e60 None None] MFA Rules not processed for user `f306ac5d4d7d429f93bbce942e92b1fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:34:26.943914 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-cbe0ddbe-c9d1-4168-abe3-80b672c55e60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:34:27.012267 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1054/2128] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:34:26 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:34:27.023565 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc4e383-75cf-4768-ac1b-439b5b9f1af6 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:34:27.025254 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-6bc4e383-75cf-4768-ac1b-439b5b9f1af6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:34:27.081579 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc4e383-75cf-4768-ac1b-439b5b9f1af6 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:34:27.085508 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7796243c-15e7-4436-8633-cb712675efa1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43bb326fb82e4d2c83fb2eaf91860dc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:34:27.086102 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7796243c-15e7-4436-8633-cb712675efa1 service cinder] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:34:27.086395 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7796243c-15e7-4436-8633-cb712675efa1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:34:27.086668 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7796243c-15e7-4436-8633-cb712675efa1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:34:27.095493 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7796243c-15e7-4436-8633-cb712675efa1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:34:27.096891 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7796243c-15e7-4436-8633-cb712675efa1 service cinder] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:34:27.106796 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1075/2129] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:34:27 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:34:27.183795 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe0ddbe-c9d1-4168-abe3-80b672c55e60 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:34:27.193440 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe0ddbe-c9d1-4168-abe3-80b672c55e60 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:34:27.197341 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab2809f-696a-49e9-b293-120518eb0ae7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43bb326fb82e4d2c83fb2eaf91860dc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:34:27.197938 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab2809f-696a-49e9-b293-120518eb0ae7 service cinder] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:34:27.198496 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab2809f-696a-49e9-b293-120518eb0ae7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:34:27.198777 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab2809f-696a-49e9-b293-120518eb0ae7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:34:27.199292 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7796243c-15e7-4436-8633-cb712675efa1 service cinder] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:34:27.200742 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-7796243c-15e7-4436-8633-cb712675efa1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:34:27.258802 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:34:27.261217 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab2809f-696a-49e9-b293-120518eb0ae7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:34:27.262803 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab2809f-696a-49e9-b293-120518eb0ae7 service cinder] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:34:27.273987 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1055/2130] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:34:27 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:34:27.284458 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7796243c-15e7-4436-8633-cb712675efa1 service cinder] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:34:27.291413 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a561ba-936d-4823-8dcc-b5638bcfa481 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e48cba3f804cf08b2477f01ae95179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:34:27.292020 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a561ba-936d-4823-8dcc-b5638bcfa481 service nova] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:34:27.292299 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a561ba-936d-4823-8dcc-b5638bcfa481 service nova] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:34:27.292571 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a561ba-936d-4823-8dcc-b5638bcfa481 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:34:27.294327 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-35a561ba-936d-4823-8dcc-b5638bcfa481 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:34:27.359328 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a561ba-936d-4823-8dcc-b5638bcfa481 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:34:27.360948 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a561ba-936d-4823-8dcc-b5638bcfa481 service nova] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:34:27.370953 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1076/2131] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:34:27 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 07:34:36.516652 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab2809f-696a-49e9-b293-120518eb0ae7 service cinder] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:34:36.580961 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab2809f-696a-49e9-b293-120518eb0ae7 service cinder] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:34:36.586205 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f79061c-2b1e-45e4-88d2-87a576b9f71b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:34:36.586866 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f79061c-2b1e-45e4-88d2-87a576b9f71b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:34:36.587143 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f79061c-2b1e-45e4-88d2-87a576b9f71b admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:34:36.587412 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f79061c-2b1e-45e4-88d2-87a576b9f71b admin admin] PATH_INFO: `/v3/users/89622fb23d2b42acb3877d4dd6817675` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:34:36.588525 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f79061c-2b1e-45e4-88d2-87a576b9f71b admin admin] RBAC: Authorizing `identity:delete_user(user_id=89622fb23d2b42acb3877d4dd6817675)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:34:36.590457 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f79061c-2b1e-45e4-88d2-87a576b9f71b admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:34:36.636441 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-6f79061c-2b1e-45e4-88d2-87a576b9f71b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '89622fb23d2b42acb3877d4dd6817675'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:34:36.643853 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-6f79061c-2b1e-45e4-88d2-87a576b9f71b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '89622fb23d2b42acb3877d4dd6817675'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:34:36.644467 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-6f79061c-2b1e-45e4-88d2-87a576b9f71b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '89622fb23d2b42acb3877d4dd6817675'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:34:36.653526 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-6f79061c-2b1e-45e4-88d2-87a576b9f71b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '89622fb23d2b42acb3877d4dd6817675'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:34:36.665619 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-6f79061c-2b1e-45e4-88d2-87a576b9f71b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '89622fb23d2b42acb3877d4dd6817675'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:34:36.671450 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1056/2132] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Nov 2 07:34:36 2022] DELETE /identity/v3/users/89622fb23d2b42acb3877d4dd6817675 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:34:36.676024 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0761eb7a-f64c-4d2d-8ee6-8915968b3849 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:34:36.676515 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0761eb7a-f64c-4d2d-8ee6-8915968b3849 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:34:36.676608 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0761eb7a-f64c-4d2d-8ee6-8915968b3849 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:34:36.683735 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:34:36.698033 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-0761eb7a-f64c-4d2d-8ee6-8915968b3849 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:34:36.699228 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-0761eb7a-f64c-4d2d-8ee6-8915968b3849 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:34:36.756510 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1077/2133] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:34:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:34:36.766745 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f79061c-2b1e-45e4-88d2-87a576b9f71b admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:34:36.768136 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-6f79061c-2b1e-45e4-88d2-87a576b9f71b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:34:36.827325 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f79061c-2b1e-45e4-88d2-87a576b9f71b admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:34:36.831222 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e239fa32-fd47-49a2-b47b-df14d37f0ea3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3ec6ecb85f94ef59762877d9ea33ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:34:36.832076 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e239fa32-fd47-49a2-b47b-df14d37f0ea3 service neutron] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:34:36.832451 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e239fa32-fd47-49a2-b47b-df14d37f0ea3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:34:36.832824 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e239fa32-fd47-49a2-b47b-df14d37f0ea3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:34:36.844182 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e239fa32-fd47-49a2-b47b-df14d37f0ea3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:34:36.845962 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e239fa32-fd47-49a2-b47b-df14d37f0ea3 service neutron] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:34:36.858926 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1057/2134] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:34:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:34:37.148601 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0761eb7a-f64c-4d2d-8ee6-8915968b3849 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:34:37.149947 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-0761eb7a-f64c-4d2d-8ee6-8915968b3849 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:34:37.152412 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c3e0d6-c38e-4f53-b60d-22504404db18 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:34:37.152801 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c3e0d6-c38e-4f53-b60d-22504404db18 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:34:37.153586 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c3e0d6-c38e-4f53-b60d-22504404db18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:34:37.162091 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0761eb7a-f64c-4d2d-8ee6-8915968b3849 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:34:37.168206 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9683275c-b956-4c36-ad33-95332173de83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:34:37.168852 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9683275c-b956-4c36-ad33-95332173de83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:34:37.169134 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9683275c-b956-4c36-ad33-95332173de83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:34:37.169404 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9683275c-b956-4c36-ad33-95332173de83 admin admin] PATH_INFO: `/v3/projects/14f1b596ab4142c8befcc0e0b878c0ed` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:34:37.170558 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9683275c-b956-4c36-ad33-95332173de83 admin admin] RBAC: Authorizing `identity:delete_project(project_id=14f1b596ab4142c8befcc0e0b878c0ed)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:34:37.172465 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9683275c-b956-4c36-ad33-95332173de83 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:34:37.175169 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-52c3e0d6-c38e-4f53-b60d-22504404db18 None None] MFA Rules not processed for user `ff4736f3d50746b0bf5d6a917b2ca0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:34:37.176556 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-52c3e0d6-c38e-4f53-b60d-22504404db18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:34:37.192320 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:34:37.227966 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-9683275c-b956-4c36-ad33-95332173de83 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '14f1b596ab4142c8befcc0e0b878c0ed', 'request_id': 'req-9683275c-b956-4c36-ad33-95332173de83'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:34:37.234502 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-9683275c-b956-4c36-ad33-95332173de83 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '14f1b596ab4142c8befcc0e0b878c0ed', 'request_id': 'req-9683275c-b956-4c36-ad33-95332173de83'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:34:37.240433 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-9683275c-b956-4c36-ad33-95332173de83 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '14f1b596ab4142c8befcc0e0b878c0ed', 'request_id': 'req-9683275c-b956-4c36-ad33-95332173de83'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:34:37.243740 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1058/2135] 10.222.0.18 () {58 vars in 1009 bytes} [Wed Nov 2 07:34:37 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:34:37.254252 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-9683275c-b956-4c36-ad33-95332173de83 admin admin] The token cache is being invalidate because project 14f1b596ab4142c8befcc0e0b878c0ed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:34:37.254252 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-9683275c-b956-4c36-ad33-95332173de83 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:34:37.254815 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-9683275c-b956-4c36-ad33-95332173de83 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:34:37.256466 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1078/2136] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Nov 2 07:34:37 2022] DELETE /identity/v3/projects/14f1b596ab4142c8befcc0e0b878c0ed => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:34:37.260366 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c3e0d6-c38e-4f53-b60d-22504404db18 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:34:37.272125 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c3e0d6-c38e-4f53-b60d-22504404db18 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:34:37.297958 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9683275c-b956-4c36-ad33-95332173de83 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:34:37.299885 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-9683275c-b956-4c36-ad33-95332173de83 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:34:37.336605 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c67091-3f3c-490b-97a8-36fefa533efe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:34:37.337240 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c67091-3f3c-490b-97a8-36fefa533efe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:34:37.337522 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c67091-3f3c-490b-97a8-36fefa533efe admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:34:37.337796 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c67091-3f3c-490b-97a8-36fefa533efe admin admin] PATH_INFO: `/v3/users/f306ac5d4d7d429f93bbce942e92b1fe` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:34:37.338951 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c67091-3f3c-490b-97a8-36fefa533efe admin admin] RBAC: Authorizing `identity:delete_user(user_id=f306ac5d4d7d429f93bbce942e92b1fe)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:34:37.341002 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c67091-3f3c-490b-97a8-36fefa533efe admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:34:37.379675 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9683275c-b956-4c36-ad33-95332173de83 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:34:37.387226 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306ebcc6-3feb-4f70-8c4c-daa8701686a4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9116f47889f4db39e0203d6cdae1593', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:34:37.387983 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306ebcc6-3feb-4f70-8c4c-daa8701686a4 service swift] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:34:37.388189 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306ebcc6-3feb-4f70-8c4c-daa8701686a4 service swift] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:34:37.388458 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306ebcc6-3feb-4f70-8c4c-daa8701686a4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:34:37.390471 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-306ebcc6-3feb-4f70-8c4c-daa8701686a4 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:34:37.396684 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-b6c67091-3f3c-490b-97a8-36fefa533efe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f306ac5d4d7d429f93bbce942e92b1fe'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:34:37.405942 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-b6c67091-3f3c-490b-97a8-36fefa533efe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f306ac5d4d7d429f93bbce942e92b1fe'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:34:37.406671 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-b6c67091-3f3c-490b-97a8-36fefa533efe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f306ac5d4d7d429f93bbce942e92b1fe'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:34:37.413846 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-b6c67091-3f3c-490b-97a8-36fefa533efe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f306ac5d4d7d429f93bbce942e92b1fe'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:34:37.420672 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:34:37.426775 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-b6c67091-3f3c-490b-97a8-36fefa533efe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f306ac5d4d7d429f93bbce942e92b1fe'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:34:37.433744 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1059/2137] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Nov 2 07:34:37 2022] DELETE /identity/v3/users/f306ac5d4d7d429f93bbce942e92b1fe => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:34:37.455991 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306ebcc6-3feb-4f70-8c4c-daa8701686a4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:34:37.457583 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306ebcc6-3feb-4f70-8c4c-daa8701686a4 service swift] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:34:37.465897 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1079/2138] 10.222.0.18 () {60 vars in 1384 bytes} [Wed Nov 2 07:34:37 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Nov 02 07:34:37.760385 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c67091-3f3c-490b-97a8-36fefa533efe admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:34:37.762163 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-b6c67091-3f3c-490b-97a8-36fefa533efe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:34:37.847421 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:34:37.874409 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c67091-3f3c-490b-97a8-36fefa533efe admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:34:37.880673 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889f5665-baf9-4bca-b5f1-f7ad02c2cf2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:34:37.881319 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889f5665-baf9-4bca-b5f1-f7ad02c2cf2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:34:37.881619 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889f5665-baf9-4bca-b5f1-f7ad02c2cf2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:34:37.881909 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889f5665-baf9-4bca-b5f1-f7ad02c2cf2c admin admin] PATH_INFO: `/v3/projects/b9680632ba4f4039a8a16feacfc7f861` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:34:37.883104 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889f5665-baf9-4bca-b5f1-f7ad02c2cf2c admin admin] RBAC: Authorizing `identity:delete_project(project_id=b9680632ba4f4039a8a16feacfc7f861)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:34:37.885013 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889f5665-baf9-4bca-b5f1-f7ad02c2cf2c admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:34:37.946210 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-889f5665-baf9-4bca-b5f1-f7ad02c2cf2c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b9680632ba4f4039a8a16feacfc7f861', 'request_id': 'req-889f5665-baf9-4bca-b5f1-f7ad02c2cf2c'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:34:37.956366 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-889f5665-baf9-4bca-b5f1-f7ad02c2cf2c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b9680632ba4f4039a8a16feacfc7f861', 'request_id': 'req-889f5665-baf9-4bca-b5f1-f7ad02c2cf2c'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:34:37.965191 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-889f5665-baf9-4bca-b5f1-f7ad02c2cf2c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b9680632ba4f4039a8a16feacfc7f861', 'request_id': 'req-889f5665-baf9-4bca-b5f1-f7ad02c2cf2c'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:34:37.979269 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-889f5665-baf9-4bca-b5f1-f7ad02c2cf2c admin admin] The token cache is being invalidate because project b9680632ba4f4039a8a16feacfc7f861 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81501) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:34:37.979853 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-889f5665-baf9-4bca-b5f1-f7ad02c2cf2c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:34:37.980599 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-889f5665-baf9-4bca-b5f1-f7ad02c2cf2c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:34:37.982503 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1060/2139] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Nov 2 07:34:37 2022] DELETE /identity/v3/projects/b9680632ba4f4039a8a16feacfc7f861 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:34:37.987197 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401923aa-5b96-40b9-a5ef-4a886fac918b None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:34:37.988066 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401923aa-5b96-40b9-a5ef-4a886fac918b None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:34:37.988831 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401923aa-5b96-40b9-a5ef-4a886fac918b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:34:38.013506 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-401923aa-5b96-40b9-a5ef-4a886fac918b None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:34:38.014795 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-401923aa-5b96-40b9-a5ef-4a886fac918b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:34:38.155351 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1080/2140] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:34:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:34:38.161196 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7215b5-d31d-4116-a936-355eee70de87 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:34:38.161595 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7215b5-d31d-4116-a936-355eee70de87 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:34:38.161997 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7215b5-d31d-4116-a936-355eee70de87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:34:38.198257 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-2e7215b5-d31d-4116-a936-355eee70de87 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:34:38.199916 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-2e7215b5-d31d-4116-a936-355eee70de87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:34:38.208954 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1061/2141] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:34:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:34:38.213743 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401923aa-5b96-40b9-a5ef-4a886fac918b None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:34:38.225922 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401923aa-5b96-40b9-a5ef-4a886fac918b None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:34:38.231700 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad74f79-99d0-4af7-af90-c52285de80e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:34:38.232938 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad74f79-99d0-4af7-af90-c52285de80e1 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:34:38.233227 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad74f79-99d0-4af7-af90-c52285de80e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:34:38.233500 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad74f79-99d0-4af7-af90-c52285de80e1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:34:38.234206 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad74f79-99d0-4af7-af90-c52285de80e1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:34:38.234536 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad74f79-99d0-4af7-af90-c52285de80e1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:34:38.236301 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad74f79-99d0-4af7-af90-c52285de80e1 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:34:38.253229 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1081/2142] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Nov 2 07:34:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:34:38.258431 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7215b5-d31d-4116-a936-355eee70de87 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:34:38.290420 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7215b5-d31d-4116-a936-355eee70de87 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:34:38.295254 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2463b431-5398-4262-927a-13c0c05e003d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:34:38.296033 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2463b431-5398-4262-927a-13c0c05e003d admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:34:38.297237 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2463b431-5398-4262-927a-13c0c05e003d admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:34:38.297661 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2463b431-5398-4262-927a-13c0c05e003d admin admin] PATH_INFO: `/v3/projects` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:34:38.298958 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2463b431-5398-4262-927a-13c0c05e003d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:34:38.301022 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2463b431-5398-4262-927a-13c0c05e003d admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:34:38.342623 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1062/2143] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Nov 2 07:34:38 2022] POST /identity/v3/projects => generated 397 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:34:38.343751 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad74f79-99d0-4af7-af90-c52285de80e1 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:34:38.356595 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad74f79-99d0-4af7-af90-c52285de80e1 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:34:38.429913 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46421563-9de9-496e-9186-5e3db020ca23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:34:38.432937 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46421563-9de9-496e-9186-5e3db020ca23 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:34:38.433216 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46421563-9de9-496e-9186-5e3db020ca23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:34:38.433493 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46421563-9de9-496e-9186-5e3db020ca23 admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:34:38.434292 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46421563-9de9-496e-9186-5e3db020ca23 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:34:38.436552 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46421563-9de9-496e-9186-5e3db020ca23 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:34:38.463537 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1082/2144] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Nov 2 07:34:38 2022] POST /identity/v3/users => generated 299 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 07:34:38.473642 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2463b431-5398-4262-927a-13c0c05e003d admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:34:38.485293 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2463b431-5398-4262-927a-13c0c05e003d admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:34:38.490566 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca38aad-ebda-4ad2-a865-f9eda4b1f9a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:34:38.491824 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca38aad-ebda-4ad2-a865-f9eda4b1f9a2 admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:34:38.492086 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca38aad-ebda-4ad2-a865-f9eda4b1f9a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:34:38.492393 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca38aad-ebda-4ad2-a865-f9eda4b1f9a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:34:38.493031 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca38aad-ebda-4ad2-a865-f9eda4b1f9a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:34:38.494806 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca38aad-ebda-4ad2-a865-f9eda4b1f9a2 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:34:38.506460 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1063/2145] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:34:38 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:34:38.510696 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46421563-9de9-496e-9186-5e3db020ca23 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:34:38.522278 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46421563-9de9-496e-9186-5e3db020ca23 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:34:38.528746 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680492d5-487a-44b3-8fb9-f0f0316ed720 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:34:38.529443 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680492d5-487a-44b3-8fb9-f0f0316ed720 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:34:38.529917 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680492d5-487a-44b3-8fb9-f0f0316ed720 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:34:38.530258 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680492d5-487a-44b3-8fb9-f0f0316ed720 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:34:38.531128 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680492d5-487a-44b3-8fb9-f0f0316ed720 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:34:38.533861 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680492d5-487a-44b3-8fb9-f0f0316ed720 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:34:38.545664 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1083/2146] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:34:38 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:34:38.549944 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca38aad-ebda-4ad2-a865-f9eda4b1f9a2 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:34:38.564775 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca38aad-ebda-4ad2-a865-f9eda4b1f9a2 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:34:38.571479 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3d9ce6-6405-4a64-b86a-f5bb46af08e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:34:38.573288 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3d9ce6-6405-4a64-b86a-f5bb46af08e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:34:38.573758 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3d9ce6-6405-4a64-b86a-f5bb46af08e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:34:38.574032 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3d9ce6-6405-4a64-b86a-f5bb46af08e7 admin admin] PATH_INFO: `/v3/projects/967ced1eeb8144bd8aebb7d28805ecad/users/aeaabfda390e4edab13002575766262b/roles/b494fedd00f24f0b84a529e8672a156a` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:34:38.591401 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3d9ce6-6405-4a64-b86a-f5bb46af08e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=967ced1eeb8144bd8aebb7d28805ecad, user_id=aeaabfda390e4edab13002575766262b, role_id=b494fedd00f24f0b84a529e8672a156a)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:34:38.594095 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3d9ce6-6405-4a64-b86a-f5bb46af08e7 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:34:38.609898 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1064/2147] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Nov 2 07:34:38 2022] PUT /identity/v3/projects/967ced1eeb8144bd8aebb7d28805ecad/users/aeaabfda390e4edab13002575766262b/roles/b494fedd00f24f0b84a529e8672a156a => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:34:38.622562 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282c2812-12ab-4067-a005-a63705f99876 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:34:38.622874 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282c2812-12ab-4067-a005-a63705f99876 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:34:38.623819 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282c2812-12ab-4067-a005-a63705f99876 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:34:38.667998 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-282c2812-12ab-4067-a005-a63705f99876 None None] MFA Rules not processed for user `aeaabfda390e4edab13002575766262b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:34:38.669620 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-282c2812-12ab-4067-a005-a63705f99876 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:34:38.691955 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:34:38.755411 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1084/2148] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:34:38 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:34:38.757290 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3d9ce6-6405-4a64-b86a-f5bb46af08e7 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:34:38.838022 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3d9ce6-6405-4a64-b86a-f5bb46af08e7 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:34:38.842856 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f56bab-6bab-41d1-91f2-2f8c843f7db8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:34:38.843536 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f56bab-6bab-41d1-91f2-2f8c843f7db8 admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:34:38.843884 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f56bab-6bab-41d1-91f2-2f8c843f7db8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:34:38.844219 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f56bab-6bab-41d1-91f2-2f8c843f7db8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:34:38.844912 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f56bab-6bab-41d1-91f2-2f8c843f7db8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:34:38.847415 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f56bab-6bab-41d1-91f2-2f8c843f7db8 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:34:38.864466 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:34:38.884838 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1065/2149] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Nov 2 07:34:38 2022] POST /identity/v3/projects => generated 399 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:34:38.888444 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282c2812-12ab-4067-a005-a63705f99876 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:34:38.902001 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282c2812-12ab-4067-a005-a63705f99876 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:34:38.967125 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003fba13-68b8-4239-9f3c-351cb85cee06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:34:38.967768 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003fba13-68b8-4239-9f3c-351cb85cee06 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:34:38.968032 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003fba13-68b8-4239-9f3c-351cb85cee06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:34:38.968306 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003fba13-68b8-4239-9f3c-351cb85cee06 admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:34:38.968953 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003fba13-68b8-4239-9f3c-351cb85cee06 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:34:38.970897 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003fba13-68b8-4239-9f3c-351cb85cee06 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:34:38.998392 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1085/2150] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Nov 2 07:34:38 2022] POST /identity/v3/users => generated 306 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:34:39.002024 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f56bab-6bab-41d1-91f2-2f8c843f7db8 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:34:39.013086 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f56bab-6bab-41d1-91f2-2f8c843f7db8 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:34:39.018149 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bea0424-80d8-4de2-bd7e-b4599147b5ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:34:39.018924 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bea0424-80d8-4de2-bd7e-b4599147b5ba admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:34:39.019210 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bea0424-80d8-4de2-bd7e-b4599147b5ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:34:39.019564 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bea0424-80d8-4de2-bd7e-b4599147b5ba admin admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:34:39.020237 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bea0424-80d8-4de2-bd7e-b4599147b5ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:34:39.021846 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bea0424-80d8-4de2-bd7e-b4599147b5ba admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:34:39.033918 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1066/2151] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:34:39 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:34:39.039117 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003fba13-68b8-4239-9f3c-351cb85cee06 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:34:39.050373 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003fba13-68b8-4239-9f3c-351cb85cee06 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:34:39.056090 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c44a29-6dbd-495f-8cdb-f3809df8eee5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:34:39.056939 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c44a29-6dbd-495f-8cdb-f3809df8eee5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:34:39.057328 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c44a29-6dbd-495f-8cdb-f3809df8eee5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:34:39.057646 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c44a29-6dbd-495f-8cdb-f3809df8eee5 admin admin] PATH_INFO: `/v3/projects/cc52eaf3fa024318948260120861e514/users/64c17c2172114a90a2ebf55cf5169fb6/roles/c4bf019fc6c44c71a6c984f6d0bab75f` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:34:39.078337 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c44a29-6dbd-495f-8cdb-f3809df8eee5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc52eaf3fa024318948260120861e514, user_id=64c17c2172114a90a2ebf55cf5169fb6, role_id=c4bf019fc6c44c71a6c984f6d0bab75f)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:34:39.081154 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c44a29-6dbd-495f-8cdb-f3809df8eee5 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:34:39.096562 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1086/2152] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Nov 2 07:34:39 2022] PUT /identity/v3/projects/cc52eaf3fa024318948260120861e514/users/64c17c2172114a90a2ebf55cf5169fb6/roles/c4bf019fc6c44c71a6c984f6d0bab75f => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:34:39.100652 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bea0424-80d8-4de2-bd7e-b4599147b5ba admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:34:39.113384 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bea0424-80d8-4de2-bd7e-b4599147b5ba admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:34:39.190765 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23acc63-3832-4fb1-94e8-eee700c80cb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:34:39.191642 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23acc63-3832-4fb1-94e8-eee700c80cb7 admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:34:39.191770 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23acc63-3832-4fb1-94e8-eee700c80cb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:34:39.192017 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23acc63-3832-4fb1-94e8-eee700c80cb7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:34:39.192582 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23acc63-3832-4fb1-94e8-eee700c80cb7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:34:39.195225 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23acc63-3832-4fb1-94e8-eee700c80cb7 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:34:39.207448 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1067/2153] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:34:39 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:34:39.211331 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c44a29-6dbd-495f-8cdb-f3809df8eee5 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:34:39.221946 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c44a29-6dbd-495f-8cdb-f3809df8eee5 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:34:39.226839 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c023a44-3920-48c5-a599-a3de245af3a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:34:39.227500 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c023a44-3920-48c5-a599-a3de245af3a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:34:39.227781 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c023a44-3920-48c5-a599-a3de245af3a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:34:39.228051 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c023a44-3920-48c5-a599-a3de245af3a9 admin admin] PATH_INFO: `/v3/projects/cc52eaf3fa024318948260120861e514/users/64c17c2172114a90a2ebf55cf5169fb6/roles/c4bf019fc6c44c71a6c984f6d0bab75f` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:34:39.231761 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c023a44-3920-48c5-a599-a3de245af3a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc52eaf3fa024318948260120861e514, user_id=64c17c2172114a90a2ebf55cf5169fb6, role_id=c4bf019fc6c44c71a6c984f6d0bab75f)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:34:39.234447 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c023a44-3920-48c5-a599-a3de245af3a9 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:34:39.247384 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1087/2154] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Nov 2 07:34:39 2022] PUT /identity/v3/projects/cc52eaf3fa024318948260120861e514/users/64c17c2172114a90a2ebf55cf5169fb6/roles/c4bf019fc6c44c71a6c984f6d0bab75f => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:34:39.258651 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec670825-a53c-4bee-8488-117b04ce4359 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:34:39.258925 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec670825-a53c-4bee-8488-117b04ce4359 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:34:39.259199 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec670825-a53c-4bee-8488-117b04ce4359 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:34:39.289151 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-ec670825-a53c-4bee-8488-117b04ce4359 None None] MFA Rules not processed for user `64c17c2172114a90a2ebf55cf5169fb6`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:34:39.290408 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-ec670825-a53c-4bee-8488-117b04ce4359 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:34:39.402487 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1068/2155] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:34:39 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:34:39.410967 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c023a44-3920-48c5-a599-a3de245af3a9 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:34:39.412917 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-9c023a44-3920-48c5-a599-a3de245af3a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:34:39.521635 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c023a44-3920-48c5-a599-a3de245af3a9 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:34:39.526011 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd6cca5-7c1f-4483-84c4-e919e9233a7b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43bb326fb82e4d2c83fb2eaf91860dc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:34:39.526655 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd6cca5-7c1f-4483-84c4-e919e9233a7b service cinder] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:34:39.526926 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd6cca5-7c1f-4483-84c4-e919e9233a7b service cinder] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:34:39.527189 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd6cca5-7c1f-4483-84c4-e919e9233a7b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:34:39.537566 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd6cca5-7c1f-4483-84c4-e919e9233a7b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:34:39.539967 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd6cca5-7c1f-4483-84c4-e919e9233a7b service cinder] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:34:39.563336 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1088/2156] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:34:39 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:34:39.641918 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec670825-a53c-4bee-8488-117b04ce4359 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:34:39.653612 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec670825-a53c-4bee-8488-117b04ce4359 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:34:39.659686 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807d779a-5c32-42fc-b839-c3a9574ae52f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43bb326fb82e4d2c83fb2eaf91860dc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:34:39.660372 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807d779a-5c32-42fc-b839-c3a9574ae52f service cinder] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:34:39.660714 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807d779a-5c32-42fc-b839-c3a9574ae52f service cinder] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:34:39.661049 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807d779a-5c32-42fc-b839-c3a9574ae52f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:34:39.729076 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807d779a-5c32-42fc-b839-c3a9574ae52f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:34:39.730586 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807d779a-5c32-42fc-b839-c3a9574ae52f service cinder] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:34:39.740796 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1069/2157] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:34:39 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:34:44.142704 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd6cca5-7c1f-4483-84c4-e919e9233a7b service cinder] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:34:44.166895 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:34:44.220761 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd6cca5-7c1f-4483-84c4-e919e9233a7b service cinder] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:34:44.226632 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ff849c-ebc9-4091-898a-ab19fc1e1efc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:34:44.227360 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ff849c-ebc9-4091-898a-ab19fc1e1efc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:34:44.227700 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ff849c-ebc9-4091-898a-ab19fc1e1efc admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:34:44.228056 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ff849c-ebc9-4091-898a-ab19fc1e1efc admin admin] PATH_INFO: `/v3/users/aeaabfda390e4edab13002575766262b` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:34:44.229648 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ff849c-ebc9-4091-898a-ab19fc1e1efc admin admin] RBAC: Authorizing `identity:delete_user(user_id=aeaabfda390e4edab13002575766262b)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:34:44.233157 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ff849c-ebc9-4091-898a-ab19fc1e1efc admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:34:44.298663 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-a0ff849c-ebc9-4091-898a-ab19fc1e1efc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aeaabfda390e4edab13002575766262b'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:34:44.320600 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-a0ff849c-ebc9-4091-898a-ab19fc1e1efc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aeaabfda390e4edab13002575766262b'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:34:44.321855 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-a0ff849c-ebc9-4091-898a-ab19fc1e1efc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aeaabfda390e4edab13002575766262b'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:34:44.334436 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-a0ff849c-ebc9-4091-898a-ab19fc1e1efc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aeaabfda390e4edab13002575766262b'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:34:44.340161 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-a0ff849c-ebc9-4091-898a-ab19fc1e1efc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aeaabfda390e4edab13002575766262b'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:34:44.365970 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1089/2158] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Nov 2 07:34:44 2022] DELETE /identity/v3/users/aeaabfda390e4edab13002575766262b => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:34:44.370770 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735b2379-3979-4ec0-8bf2-c4a4bd08761e None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:34:44.371263 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735b2379-3979-4ec0-8bf2-c4a4bd08761e None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:34:44.371346 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735b2379-3979-4ec0-8bf2-c4a4bd08761e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:34:44.397383 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:34:44.402754 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-735b2379-3979-4ec0-8bf2-c4a4bd08761e None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:34:44.404854 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-735b2379-3979-4ec0-8bf2-c4a4bd08761e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:34:44.507679 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1070/2159] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:34:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:34:44.523777 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ff849c-ebc9-4091-898a-ab19fc1e1efc admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:34:44.525509 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-a0ff849c-ebc9-4091-898a-ab19fc1e1efc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:34:44.601415 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ff849c-ebc9-4091-898a-ab19fc1e1efc admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:34:44.605289 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed3c36e-9c5e-459b-bfed-cf564bab97b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3ec6ecb85f94ef59762877d9ea33ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:34:44.605932 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed3c36e-9c5e-459b-bfed-cf564bab97b8 service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:34:44.606206 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed3c36e-9c5e-459b-bfed-cf564bab97b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:34:44.606537 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed3c36e-9c5e-459b-bfed-cf564bab97b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:34:44.618094 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed3c36e-9c5e-459b-bfed-cf564bab97b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:34:44.619619 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed3c36e-9c5e-459b-bfed-cf564bab97b8 service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:34:44.634390 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1090/2160] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:34:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:34:45.058143 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735b2379-3979-4ec0-8bf2-c4a4bd08761e None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:34:45.059789 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-735b2379-3979-4ec0-8bf2-c4a4bd08761e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:34:45.091457 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735b2379-3979-4ec0-8bf2-c4a4bd08761e None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:34:45.096747 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edd3344-7ed1-4870-a576-104e2781af78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:34:45.097457 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edd3344-7ed1-4870-a576-104e2781af78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:34:45.097805 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edd3344-7ed1-4870-a576-104e2781af78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:34:45.098149 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edd3344-7ed1-4870-a576-104e2781af78 admin admin] PATH_INFO: `/v3/projects/967ced1eeb8144bd8aebb7d28805ecad` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:34:45.099577 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edd3344-7ed1-4870-a576-104e2781af78 admin admin] RBAC: Authorizing `identity:delete_project(project_id=967ced1eeb8144bd8aebb7d28805ecad)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:34:45.102408 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edd3344-7ed1-4870-a576-104e2781af78 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:34:45.163768 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-7edd3344-7ed1-4870-a576-104e2781af78 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '967ced1eeb8144bd8aebb7d28805ecad', 'request_id': 'req-7edd3344-7ed1-4870-a576-104e2781af78'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:34:45.170469 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-7edd3344-7ed1-4870-a576-104e2781af78 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '967ced1eeb8144bd8aebb7d28805ecad', 'request_id': 'req-7edd3344-7ed1-4870-a576-104e2781af78'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:34:45.183422 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-7edd3344-7ed1-4870-a576-104e2781af78 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '967ced1eeb8144bd8aebb7d28805ecad', 'request_id': 'req-7edd3344-7ed1-4870-a576-104e2781af78'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:34:45.195207 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-7edd3344-7ed1-4870-a576-104e2781af78 admin admin] The token cache is being invalidate because project 967ced1eeb8144bd8aebb7d28805ecad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81501) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:34:45.195490 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-7edd3344-7ed1-4870-a576-104e2781af78 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:34:45.196137 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-7edd3344-7ed1-4870-a576-104e2781af78 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:34:45.197942 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1071/2161] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Nov 2 07:34:45 2022] DELETE /identity/v3/projects/967ced1eeb8144bd8aebb7d28805ecad => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:34:45.204247 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed3c36e-9c5e-459b-bfed-cf564bab97b8 service neutron] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:34:45.215911 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed3c36e-9c5e-459b-bfed-cf564bab97b8 service neutron] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:34:45.262879 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:34:45.290470 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d23e81-a99a-4166-b25e-6df55b320fdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:34:45.291190 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d23e81-a99a-4166-b25e-6df55b320fdc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:34:45.291374 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d23e81-a99a-4166-b25e-6df55b320fdc admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:34:45.291649 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d23e81-a99a-4166-b25e-6df55b320fdc admin admin] PATH_INFO: `/v3/users/64c17c2172114a90a2ebf55cf5169fb6` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:34:45.292856 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d23e81-a99a-4166-b25e-6df55b320fdc admin admin] RBAC: Authorizing `identity:delete_user(user_id=64c17c2172114a90a2ebf55cf5169fb6)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:34:45.294949 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d23e81-a99a-4166-b25e-6df55b320fdc admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:34:45.360219 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-26d23e81-a99a-4166-b25e-6df55b320fdc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '64c17c2172114a90a2ebf55cf5169fb6'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:34:45.372463 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-26d23e81-a99a-4166-b25e-6df55b320fdc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '64c17c2172114a90a2ebf55cf5169fb6'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:34:45.373445 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-26d23e81-a99a-4166-b25e-6df55b320fdc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '64c17c2172114a90a2ebf55cf5169fb6'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:34:45.381791 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-26d23e81-a99a-4166-b25e-6df55b320fdc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '64c17c2172114a90a2ebf55cf5169fb6'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:34:45.382582 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-26d23e81-a99a-4166-b25e-6df55b320fdc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '64c17c2172114a90a2ebf55cf5169fb6'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:34:45.406331 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1091/2162] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Nov 2 07:34:45 2022] DELETE /identity/v3/users/64c17c2172114a90a2ebf55cf5169fb6 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:34:45.809055 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edd3344-7ed1-4870-a576-104e2781af78 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:34:45.812502 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-7edd3344-7ed1-4870-a576-104e2781af78 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:34:45.830327 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edd3344-7ed1-4870-a576-104e2781af78 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:34:45.905761 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6c96fd-7784-4fe3-9a9c-79acc3bd58eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:34:45.906704 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6c96fd-7784-4fe3-9a9c-79acc3bd58eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:34:45.907047 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6c96fd-7784-4fe3-9a9c-79acc3bd58eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:34:45.907383 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6c96fd-7784-4fe3-9a9c-79acc3bd58eb admin admin] PATH_INFO: `/v3/projects/cc52eaf3fa024318948260120861e514` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:34:45.908587 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6c96fd-7784-4fe3-9a9c-79acc3bd58eb admin admin] RBAC: Authorizing `identity:delete_project(project_id=cc52eaf3fa024318948260120861e514)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:34:45.910672 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6c96fd-7784-4fe3-9a9c-79acc3bd58eb admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:34:45.953392 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:34:45.985752 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-4f6c96fd-7784-4fe3-9a9c-79acc3bd58eb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cc52eaf3fa024318948260120861e514', 'request_id': 'req-4f6c96fd-7784-4fe3-9a9c-79acc3bd58eb'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:34:45.993793 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-4f6c96fd-7784-4fe3-9a9c-79acc3bd58eb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cc52eaf3fa024318948260120861e514', 'request_id': 'req-4f6c96fd-7784-4fe3-9a9c-79acc3bd58eb'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:34:46.002609 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-4f6c96fd-7784-4fe3-9a9c-79acc3bd58eb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cc52eaf3fa024318948260120861e514', 'request_id': 'req-4f6c96fd-7784-4fe3-9a9c-79acc3bd58eb'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:34:46.013865 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-4f6c96fd-7784-4fe3-9a9c-79acc3bd58eb admin admin] The token cache is being invalidate because project cc52eaf3fa024318948260120861e514 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81501) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:34:46.014159 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-4f6c96fd-7784-4fe3-9a9c-79acc3bd58eb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:34:46.014899 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-4f6c96fd-7784-4fe3-9a9c-79acc3bd58eb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:34:46.016738 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1072/2163] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Nov 2 07:34:45 2022] DELETE /identity/v3/projects/cc52eaf3fa024318948260120861e514 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:34:46.021492 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867c5713-7bdc-424e-8e33-0af8471b6350 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:34:46.021780 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867c5713-7bdc-424e-8e33-0af8471b6350 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:34:46.022078 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867c5713-7bdc-424e-8e33-0af8471b6350 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:34:46.045477 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-867c5713-7bdc-424e-8e33-0af8471b6350 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:34:46.046768 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-867c5713-7bdc-424e-8e33-0af8471b6350 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:34:46.132898 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1092/2164] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:34:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:34:46.140804 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa172a45-f050-4df4-96a8-7299153904dc None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:34:46.141088 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa172a45-f050-4df4-96a8-7299153904dc None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:34:46.141362 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa172a45-f050-4df4-96a8-7299153904dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:34:46.164781 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-aa172a45-f050-4df4-96a8-7299153904dc None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:34:46.166244 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-aa172a45-f050-4df4-96a8-7299153904dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:34:46.174665 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1073/2165] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:34:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:34:46.179654 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867c5713-7bdc-424e-8e33-0af8471b6350 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:34:46.194915 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867c5713-7bdc-424e-8e33-0af8471b6350 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:34:46.221720 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b7013c-eba6-4001-9bd8-96a61f7dc16e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:34:46.222740 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b7013c-eba6-4001-9bd8-96a61f7dc16e admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:34:46.223348 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b7013c-eba6-4001-9bd8-96a61f7dc16e admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:34:46.223807 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b7013c-eba6-4001-9bd8-96a61f7dc16e admin admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:34:46.224715 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b7013c-eba6-4001-9bd8-96a61f7dc16e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:34:46.225016 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b7013c-eba6-4001-9bd8-96a61f7dc16e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:34:46.227275 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b7013c-eba6-4001-9bd8-96a61f7dc16e admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:34:46.245276 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1093/2166] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Nov 2 07:34:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:34:46.246553 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa172a45-f050-4df4-96a8-7299153904dc None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:34:46.257945 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa172a45-f050-4df4-96a8-7299153904dc None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:34:46.263202 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cf18a3-0733-4171-a478-272efc6295c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:34:46.263913 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cf18a3-0733-4171-a478-272efc6295c2 admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:34:46.264981 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cf18a3-0733-4171-a478-272efc6295c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:34:46.265335 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cf18a3-0733-4171-a478-272efc6295c2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:34:46.266048 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11cf18a3-0733-4171-a478-272efc6295c2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:34:46.268071 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11cf18a3-0733-4171-a478-272efc6295c2 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:34:46.310677 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1074/2167] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Nov 2 07:34:46 2022] POST /identity/v3/projects => generated 365 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 07:34:46.315488 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b7013c-eba6-4001-9bd8-96a61f7dc16e admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:34:46.327312 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b7013c-eba6-4001-9bd8-96a61f7dc16e admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:34:46.390137 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:34:46.409677 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15126873-1d97-4c9b-8256-584c91372e1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:34:46.410427 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15126873-1d97-4c9b-8256-584c91372e1e admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:34:46.410788 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15126873-1d97-4c9b-8256-584c91372e1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:34:46.411145 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15126873-1d97-4c9b-8256-584c91372e1e admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:34:46.411942 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15126873-1d97-4c9b-8256-584c91372e1e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:34:46.414563 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15126873-1d97-4c9b-8256-584c91372e1e admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:34:46.443196 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1094/2168] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Nov 2 07:34:46 2022] POST /identity/v3/users => generated 283 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:34:46.444236 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cf18a3-0733-4171-a478-272efc6295c2 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:34:46.458222 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cf18a3-0733-4171-a478-272efc6295c2 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:34:46.466746 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd509e4c-31a4-4947-9c7d-eb18b28e16c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:34:46.467527 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd509e4c-31a4-4947-9c7d-eb18b28e16c5 admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:34:46.467900 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd509e4c-31a4-4947-9c7d-eb18b28e16c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:34:46.468234 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd509e4c-31a4-4947-9c7d-eb18b28e16c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:34:46.468953 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd509e4c-31a4-4947-9c7d-eb18b28e16c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:34:46.470657 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd509e4c-31a4-4947-9c7d-eb18b28e16c5 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:34:46.482446 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1075/2169] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:34:46 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 07:34:46.486808 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15126873-1d97-4c9b-8256-584c91372e1e admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:34:46.502407 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15126873-1d97-4c9b-8256-584c91372e1e admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:34:46.505182 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebed8c1-34be-479b-b2a6-e7b93b278099 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:34:46.507037 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebed8c1-34be-479b-b2a6-e7b93b278099 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:34:46.507729 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebed8c1-34be-479b-b2a6-e7b93b278099 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:34:46.508000 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebed8c1-34be-479b-b2a6-e7b93b278099 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:34:46.508659 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebed8c1-34be-479b-b2a6-e7b93b278099 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:34:46.510257 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebed8c1-34be-479b-b2a6-e7b93b278099 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:34:46.530406 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1095/2170] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:34:46 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:34:46.533012 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd509e4c-31a4-4947-9c7d-eb18b28e16c5 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:34:46.552426 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd509e4c-31a4-4947-9c7d-eb18b28e16c5 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:34:46.558111 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e27ed12-ea34-4b07-8806-0a1c7d616ab2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:34:46.558795 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e27ed12-ea34-4b07-8806-0a1c7d616ab2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:34:46.559185 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e27ed12-ea34-4b07-8806-0a1c7d616ab2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:34:46.559460 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e27ed12-ea34-4b07-8806-0a1c7d616ab2 admin admin] PATH_INFO: `/v3/projects/6672680f45eb4fcba7d107518b7a8796/users/0d4e935d761b427cbc209e1d3cd6f781/roles/b494fedd00f24f0b84a529e8672a156a` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:34:46.579126 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e27ed12-ea34-4b07-8806-0a1c7d616ab2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6672680f45eb4fcba7d107518b7a8796, user_id=0d4e935d761b427cbc209e1d3cd6f781, role_id=b494fedd00f24f0b84a529e8672a156a)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:34:46.583072 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e27ed12-ea34-4b07-8806-0a1c7d616ab2 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:34:46.611587 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1076/2171] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Nov 2 07:34:46 2022] PUT /identity/v3/projects/6672680f45eb4fcba7d107518b7a8796/users/0d4e935d761b427cbc209e1d3cd6f781/roles/b494fedd00f24f0b84a529e8672a156a => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:34:46.613742 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9929f3e7-f4a9-455b-b265-9d4fd3d9100c None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:34:46.614028 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9929f3e7-f4a9-455b-b265-9d4fd3d9100c None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:34:46.614298 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9929f3e7-f4a9-455b-b265-9d4fd3d9100c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:34:46.658676 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-9929f3e7-f4a9-455b-b265-9d4fd3d9100c None None] MFA Rules not processed for user `0d4e935d761b427cbc209e1d3cd6f781`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:34:46.659902 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-9929f3e7-f4a9-455b-b265-9d4fd3d9100c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:34:46.751497 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1096/2172] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:34:46 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:34:46.757400 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e27ed12-ea34-4b07-8806-0a1c7d616ab2 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:34:46.834654 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e27ed12-ea34-4b07-8806-0a1c7d616ab2 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:34:46.842131 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7547b35e-0185-411d-970d-d5b73191a183 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:34:46.842794 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7547b35e-0185-411d-970d-d5b73191a183 admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:34:46.843075 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7547b35e-0185-411d-970d-d5b73191a183 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:34:46.843347 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7547b35e-0185-411d-970d-d5b73191a183 admin admin] PATH_INFO: `/v3/projects` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:34:46.843997 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7547b35e-0185-411d-970d-d5b73191a183 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:34:46.846128 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7547b35e-0185-411d-970d-d5b73191a183 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:34:46.886247 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1077/2173] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Nov 2 07:34:46 2022] POST /identity/v3/projects => generated 367 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:34:46.888354 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9929f3e7-f4a9-455b-b265-9d4fd3d9100c None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:34:46.900078 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9929f3e7-f4a9-455b-b265-9d4fd3d9100c None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:34:46.974166 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980c4cb1-2fbe-4e88-b47e-55139c0eb06c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:34:46.975047 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980c4cb1-2fbe-4e88-b47e-55139c0eb06c admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:34:46.975328 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980c4cb1-2fbe-4e88-b47e-55139c0eb06c admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:34:46.976246 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980c4cb1-2fbe-4e88-b47e-55139c0eb06c admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:34:46.977143 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980c4cb1-2fbe-4e88-b47e-55139c0eb06c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:34:46.979346 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980c4cb1-2fbe-4e88-b47e-55139c0eb06c admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:34:47.007603 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1097/2174] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Nov 2 07:34:46 2022] POST /identity/v3/users => generated 290 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:34:47.011122 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7547b35e-0185-411d-970d-d5b73191a183 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:34:47.025199 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7547b35e-0185-411d-970d-d5b73191a183 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:34:47.031024 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474e8022-8760-4001-a501-85874acf137c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:34:47.031652 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474e8022-8760-4001-a501-85874acf137c admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:34:47.031928 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474e8022-8760-4001-a501-85874acf137c admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:34:47.032204 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474e8022-8760-4001-a501-85874acf137c admin admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:34:47.032990 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474e8022-8760-4001-a501-85874acf137c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:34:47.035534 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474e8022-8760-4001-a501-85874acf137c admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:34:47.066710 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1078/2175] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:34:47 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 07:34:47.067998 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980c4cb1-2fbe-4e88-b47e-55139c0eb06c admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:34:47.087375 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980c4cb1-2fbe-4e88-b47e-55139c0eb06c admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:34:47.100105 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90708712-5788-48b6-8076-e05f03c95e8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:34:47.100860 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90708712-5788-48b6-8076-e05f03c95e8e admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:34:47.101143 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90708712-5788-48b6-8076-e05f03c95e8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:34:47.101415 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90708712-5788-48b6-8076-e05f03c95e8e admin admin] PATH_INFO: `/v3/projects/2146095387254dd58bf0dc6ed1193957/users/aeeb5d01f71c4f8989969d8f4af8606b/roles/c4bf019fc6c44c71a6c984f6d0bab75f` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:34:47.126549 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90708712-5788-48b6-8076-e05f03c95e8e admin admin] RBAC: Authorizing `identity:create_grant(project_id=2146095387254dd58bf0dc6ed1193957, user_id=aeeb5d01f71c4f8989969d8f4af8606b, role_id=c4bf019fc6c44c71a6c984f6d0bab75f)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:34:47.128314 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90708712-5788-48b6-8076-e05f03c95e8e admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:34:47.143710 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1098/2176] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Nov 2 07:34:47 2022] PUT /identity/v3/projects/2146095387254dd58bf0dc6ed1193957/users/aeeb5d01f71c4f8989969d8f4af8606b/roles/c4bf019fc6c44c71a6c984f6d0bab75f => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 07:34:47.148162 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474e8022-8760-4001-a501-85874acf137c admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:34:47.161695 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474e8022-8760-4001-a501-85874acf137c admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:34:47.192017 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:34:47.229028 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b628976-f096-47f7-aa53-2e7e67de7333 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:34:47.230049 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b628976-f096-47f7-aa53-2e7e67de7333 admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:34:47.230498 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b628976-f096-47f7-aa53-2e7e67de7333 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:34:47.230775 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b628976-f096-47f7-aa53-2e7e67de7333 admin admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:34:47.231577 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b628976-f096-47f7-aa53-2e7e67de7333 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:34:47.233187 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b628976-f096-47f7-aa53-2e7e67de7333 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:34:47.246390 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1079/2177] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:34:47 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 07:34:47.248964 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90708712-5788-48b6-8076-e05f03c95e8e admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:34:47.261173 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90708712-5788-48b6-8076-e05f03c95e8e admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:34:47.266989 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a947c1-847c-4d78-bd0a-112dbb3b4bac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:34:47.267647 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a947c1-847c-4d78-bd0a-112dbb3b4bac admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:34:47.267925 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a947c1-847c-4d78-bd0a-112dbb3b4bac admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:34:47.268197 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a947c1-847c-4d78-bd0a-112dbb3b4bac admin admin] PATH_INFO: `/v3/projects/2146095387254dd58bf0dc6ed1193957/users/aeeb5d01f71c4f8989969d8f4af8606b/roles/c4bf019fc6c44c71a6c984f6d0bab75f` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:34:47.274759 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a947c1-847c-4d78-bd0a-112dbb3b4bac admin admin] RBAC: Authorizing `identity:create_grant(project_id=2146095387254dd58bf0dc6ed1193957, user_id=aeeb5d01f71c4f8989969d8f4af8606b, role_id=c4bf019fc6c44c71a6c984f6d0bab75f)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:34:47.277659 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a947c1-847c-4d78-bd0a-112dbb3b4bac admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:34:47.294431 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1099/2178] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Nov 2 07:34:47 2022] PUT /identity/v3/projects/2146095387254dd58bf0dc6ed1193957/users/aeeb5d01f71c4f8989969d8f4af8606b/roles/c4bf019fc6c44c71a6c984f6d0bab75f => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:34:47.301917 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998c0a37-df9b-4ab1-a6ce-2ea3bd120c87 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:34:47.302202 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998c0a37-df9b-4ab1-a6ce-2ea3bd120c87 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:34:47.302510 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998c0a37-df9b-4ab1-a6ce-2ea3bd120c87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:34:47.345053 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-998c0a37-df9b-4ab1-a6ce-2ea3bd120c87 None None] MFA Rules not processed for user `aeeb5d01f71c4f8989969d8f4af8606b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:34:47.346202 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-998c0a37-df9b-4ab1-a6ce-2ea3bd120c87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:34:47.420728 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1080/2179] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:34:47 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:34:47.433100 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a947c1-847c-4d78-bd0a-112dbb3b4bac admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:34:47.435321 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-42a947c1-847c-4d78-bd0a-112dbb3b4bac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:34:47.445293 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:34:47.500575 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a947c1-847c-4d78-bd0a-112dbb3b4bac admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:34:47.504580 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2e0944-d7fe-4dfd-b012-8a49474fe377 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43bb326fb82e4d2c83fb2eaf91860dc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:34:47.505459 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2e0944-d7fe-4dfd-b012-8a49474fe377 service cinder] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:34:47.505459 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2e0944-d7fe-4dfd-b012-8a49474fe377 service cinder] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:34:47.505724 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2e0944-d7fe-4dfd-b012-8a49474fe377 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:34:47.515625 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2e0944-d7fe-4dfd-b012-8a49474fe377 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:34:47.517060 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2e0944-d7fe-4dfd-b012-8a49474fe377 service cinder] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:34:47.529296 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1100/2180] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:34:47 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:34:47.629852 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998c0a37-df9b-4ab1-a6ce-2ea3bd120c87 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:34:47.640899 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998c0a37-df9b-4ab1-a6ce-2ea3bd120c87 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:34:47.644456 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4618b7a3-5f95-4384-9a26-ebde812828f9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43bb326fb82e4d2c83fb2eaf91860dc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:34:47.645062 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4618b7a3-5f95-4384-9a26-ebde812828f9 service cinder] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:34:47.645345 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4618b7a3-5f95-4384-9a26-ebde812828f9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:34:47.645619 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4618b7a3-5f95-4384-9a26-ebde812828f9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:34:47.716462 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4618b7a3-5f95-4384-9a26-ebde812828f9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:34:47.717941 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4618b7a3-5f95-4384-9a26-ebde812828f9 service cinder] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:34:47.727530 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1081/2181] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:34:47 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 07:34:47.865417 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2e0944-d7fe-4dfd-b012-8a49474fe377 service cinder] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:34:47.867705 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-0a2e0944-d7fe-4dfd-b012-8a49474fe377 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:34:47.993365 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2e0944-d7fe-4dfd-b012-8a49474fe377 service cinder] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:34:47.998994 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2692de-0ebe-444d-a71a-a7e529e43631 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3ec6ecb85f94ef59762877d9ea33ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:34:47.999783 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2692de-0ebe-444d-a71a-a7e529e43631 service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:34:48.000065 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2692de-0ebe-444d-a71a-a7e529e43631 service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:34:48.000735 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2692de-0ebe-444d-a71a-a7e529e43631 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:34:48.003893 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-7b2692de-0ebe-444d-a71a-a7e529e43631 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:34:48.020370 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2692de-0ebe-444d-a71a-a7e529e43631 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:34:48.021828 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2692de-0ebe-444d-a71a-a7e529e43631 service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:34:48.040690 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1101/2182] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:34:47 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:34:52.114016 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ae3d32-0e86-4a39-adfd-1c791f276c1d None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:34:52.114874 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ae3d32-0e86-4a39-adfd-1c791f276c1d None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:34:52.115342 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ae3d32-0e86-4a39-adfd-1c791f276c1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:34:52.135688 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-d6ae3d32-0e86-4a39-adfd-1c791f276c1d None None] MFA Rules not processed for user `47965dc7f5964bbfa4efff0822530965`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:34:52.136959 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-d6ae3d32-0e86-4a39-adfd-1c791f276c1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:34:52.179907 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:34:52.194058 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1082/2183] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:34:52 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:34:52.506205 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2692de-0ebe-444d-a71a-a7e529e43631 service neutron] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:34:52.508631 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-7b2692de-0ebe-444d-a71a-a7e529e43631 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:34:52.518872 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2692de-0ebe-444d-a71a-a7e529e43631 service neutron] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:34:52.522987 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba70807-4a35-4496-acbc-e413f7e39bd2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3ec6ecb85f94ef59762877d9ea33ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:34:52.523384 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba70807-4a35-4496-acbc-e413f7e39bd2 service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:34:52.523668 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba70807-4a35-4496-acbc-e413f7e39bd2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:34:52.523947 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba70807-4a35-4496-acbc-e413f7e39bd2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:34:52.525646 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-dba70807-4a35-4496-acbc-e413f7e39bd2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:34:52.536166 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba70807-4a35-4496-acbc-e413f7e39bd2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:34:52.537576 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba70807-4a35-4496-acbc-e413f7e39bd2 service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:34:52.546707 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1102/2184] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:34:52 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:34:52.875263 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c049be0-d691-4c85-b78c-f536685f4056 None None] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:34:52.875724 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c049be0-d691-4c85-b78c-f536685f4056 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:34:52.876097 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c049be0-d691-4c85-b78c-f536685f4056 None None] PATH_INFO: `/` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:34:52.878121 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1083/2185] 10.222.0.18 () {54 vars in 921 bytes} [Wed Nov 2 07:34:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:34:52.886135 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e99085-a55b-4f29-a272-ba946381c6d0 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:34:52.886809 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e99085-a55b-4f29-a272-ba946381c6d0 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:34:52.887262 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e99085-a55b-4f29-a272-ba946381c6d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:34:52.915038 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c049be0-d691-4c85-b78c-f536685f4056 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:34:52.916978 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-8c049be0-d691-4c85-b78c-f536685f4056 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:34:52.926705 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-72e99085-a55b-4f29-a272-ba946381c6d0 None None] MFA Rules not processed for user `69e48cba3f804cf08b2477f01ae95179`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:34:52.928094 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-72e99085-a55b-4f29-a272-ba946381c6d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:34:53.036581 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c049be0-d691-4c85-b78c-f536685f4056 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:34:53.043493 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1103/2186] 10.222.0.18 () {58 vars in 1018 bytes} [Wed Nov 2 07:34:52 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:34:53.044309 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e99085-a55b-4f29-a272-ba946381c6d0 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:34:53.047996 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6e69ad-f0b6-463c-b517-1d52bf1e7c9d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2fc6a156c54d0c9d325901d0f7f6fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:34:53.048798 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6e69ad-f0b6-463c-b517-1d52bf1e7c9d service placement] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:34:53.049303 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6e69ad-f0b6-463c-b517-1d52bf1e7c9d service placement] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:34:53.049696 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6e69ad-f0b6-463c-b517-1d52bf1e7c9d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:34:53.052309 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-fc6e69ad-f0b6-463c-b517-1d52bf1e7c9d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:34:53.066650 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6e69ad-f0b6-463c-b517-1d52bf1e7c9d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:34:53.068322 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6e69ad-f0b6-463c-b517-1d52bf1e7c9d service placement] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:34:53.085178 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:34:53.089093 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1084/2187] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:34:52 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:34:53.090792 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6e69ad-f0b6-463c-b517-1d52bf1e7c9d service placement] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:34:53.092443 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-fc6e69ad-f0b6-463c-b517-1d52bf1e7c9d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:34:53.118252 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e99085-a55b-4f29-a272-ba946381c6d0 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:34:53.123091 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f56fcae-70a0-467f-accc-6364e1b82e15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:34:53.123828 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f56fcae-70a0-467f-accc-6364e1b82e15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:34:53.124116 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f56fcae-70a0-467f-accc-6364e1b82e15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:34:53.124392 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f56fcae-70a0-467f-accc-6364e1b82e15 admin admin] PATH_INFO: `/v3/users/0d4e935d761b427cbc209e1d3cd6f781` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:34:53.125613 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f56fcae-70a0-467f-accc-6364e1b82e15 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0d4e935d761b427cbc209e1d3cd6f781)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:34:53.127858 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f56fcae-70a0-467f-accc-6364e1b82e15 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:34:53.155453 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6e69ad-f0b6-463c-b517-1d52bf1e7c9d service placement] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:34:53.159869 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16a0c9e-eac7-481a-ad41-2d0676621bca service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5353f2af0f44d668e2a269647aa1dc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:34:53.160479 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16a0c9e-eac7-481a-ad41-2d0676621bca service glance] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:34:53.160898 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16a0c9e-eac7-481a-ad41-2d0676621bca service glance] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:34:53.161325 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16a0c9e-eac7-481a-ad41-2d0676621bca service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:34:53.174226 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16a0c9e-eac7-481a-ad41-2d0676621bca service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:34:53.175912 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16a0c9e-eac7-481a-ad41-2d0676621bca service glance] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:34:53.189984 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1085/2188] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:34:53 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:34:53.198838 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-8f56fcae-70a0-467f-accc-6364e1b82e15 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0d4e935d761b427cbc209e1d3cd6f781'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:34:53.208961 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-8f56fcae-70a0-467f-accc-6364e1b82e15 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0d4e935d761b427cbc209e1d3cd6f781'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:34:53.210083 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-8f56fcae-70a0-467f-accc-6364e1b82e15 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0d4e935d761b427cbc209e1d3cd6f781'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:34:53.218864 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-8f56fcae-70a0-467f-accc-6364e1b82e15 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0d4e935d761b427cbc209e1d3cd6f781'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:34:53.219799 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-8f56fcae-70a0-467f-accc-6364e1b82e15 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0d4e935d761b427cbc209e1d3cd6f781'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:34:53.240819 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1104/2189] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Nov 2 07:34:53 2022] DELETE /identity/v3/users/0d4e935d761b427cbc209e1d3cd6f781 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:34:53.246912 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afb0034-2b2d-4730-8e94-90ab26df509f None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:34:53.247324 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afb0034-2b2d-4730-8e94-90ab26df509f None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:34:53.247584 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afb0034-2b2d-4730-8e94-90ab26df509f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:34:53.271355 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-5afb0034-2b2d-4730-8e94-90ab26df509f None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:34:53.273736 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c0a623-b0af-4a3f-bbf8-d9668bd5c1ef None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:34:53.274019 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c0a623-b0af-4a3f-bbf8-d9668bd5c1ef None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:34:53.274289 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c0a623-b0af-4a3f-bbf8-d9668bd5c1ef None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:34:53.275661 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1105/2190] 10.222.0.18 () {54 vars in 921 bytes} [Wed Nov 2 07:34:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:34:53.276571 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-5afb0034-2b2d-4730-8e94-90ab26df509f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:34:53.280856 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e15c75-829b-4548-808c-968e10748e94 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:34:53.281143 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e15c75-829b-4548-808c-968e10748e94 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:34:53.281410 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e15c75-829b-4548-808c-968e10748e94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:34:53.310614 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-84e15c75-829b-4548-808c-968e10748e94 None None] MFA Rules not processed for user `b3ec6ecb85f94ef59762877d9ea33ed0`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:34:53.311846 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-84e15c75-829b-4548-808c-968e10748e94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:34:53.362324 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1086/2191] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:34:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:34:53.379475 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afb0034-2b2d-4730-8e94-90ab26df509f None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:34:53.380992 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-5afb0034-2b2d-4730-8e94-90ab26df509f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:34:53.393535 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afb0034-2b2d-4730-8e94-90ab26df509f None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:34:53.396436 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1106/2192] 10.222.0.18 () {58 vars in 1018 bytes} [Wed Nov 2 07:34:53 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:34:53.398311 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f9c7d7-2a5c-4297-9edf-05182a4a891b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3ec6ecb85f94ef59762877d9ea33ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:34:53.398942 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f9c7d7-2a5c-4297-9edf-05182a4a891b service neutron] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:34:53.399223 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f9c7d7-2a5c-4297-9edf-05182a4a891b service neutron] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:34:53.399489 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f9c7d7-2a5c-4297-9edf-05182a4a891b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:34:53.409226 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f9c7d7-2a5c-4297-9edf-05182a4a891b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:34:53.410706 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f9c7d7-2a5c-4297-9edf-05182a4a891b service neutron] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:34:53.411609 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e15c75-829b-4548-808c-968e10748e94 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:34:53.413132 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-84e15c75-829b-4548-808c-968e10748e94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:34:53.421444 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1087/2193] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:34:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:34:53.424970 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e15c75-829b-4548-808c-968e10748e94 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:34:53.430967 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628d1d18-2a28-4e22-9169-bd617cd32e76 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3ec6ecb85f94ef59762877d9ea33ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:34:53.431539 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628d1d18-2a28-4e22-9169-bd617cd32e76 service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:34:53.432164 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628d1d18-2a28-4e22-9169-bd617cd32e76 service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:34:53.432290 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628d1d18-2a28-4e22-9169-bd617cd32e76 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:34:53.441338 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628d1d18-2a28-4e22-9169-bd617cd32e76 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:34:53.443004 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628d1d18-2a28-4e22-9169-bd617cd32e76 service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:34:53.454753 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1107/2194] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:34:53 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 07:34:53.746253 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f9c7d7-2a5c-4297-9edf-05182a4a891b service neutron] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:34:53.748142 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-98f9c7d7-2a5c-4297-9edf-05182a4a891b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:34:53.782005 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f9c7d7-2a5c-4297-9edf-05182a4a891b service neutron] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:34:53.787928 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fefd92-5e57-4cd6-8dc1-0199d94ca1c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:34:53.789769 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fefd92-5e57-4cd6-8dc1-0199d94ca1c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:34:53.790049 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fefd92-5e57-4cd6-8dc1-0199d94ca1c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:34:53.790318 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fefd92-5e57-4cd6-8dc1-0199d94ca1c9 admin admin] PATH_INFO: `/v3/projects/6672680f45eb4fcba7d107518b7a8796` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:34:53.791469 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fefd92-5e57-4cd6-8dc1-0199d94ca1c9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6672680f45eb4fcba7d107518b7a8796)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:34:53.794017 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fefd92-5e57-4cd6-8dc1-0199d94ca1c9 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:34:53.801424 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:34:53.854549 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-e4fefd92-5e57-4cd6-8dc1-0199d94ca1c9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6672680f45eb4fcba7d107518b7a8796', 'request_id': 'req-e4fefd92-5e57-4cd6-8dc1-0199d94ca1c9'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:34:53.866031 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-e4fefd92-5e57-4cd6-8dc1-0199d94ca1c9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6672680f45eb4fcba7d107518b7a8796', 'request_id': 'req-e4fefd92-5e57-4cd6-8dc1-0199d94ca1c9'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:34:53.876120 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-e4fefd92-5e57-4cd6-8dc1-0199d94ca1c9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6672680f45eb4fcba7d107518b7a8796', 'request_id': 'req-e4fefd92-5e57-4cd6-8dc1-0199d94ca1c9'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:34:53.888442 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-e4fefd92-5e57-4cd6-8dc1-0199d94ca1c9 admin admin] The token cache is being invalidate because project 6672680f45eb4fcba7d107518b7a8796 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81501) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:34:53.888792 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-e4fefd92-5e57-4cd6-8dc1-0199d94ca1c9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:34:53.889692 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-e4fefd92-5e57-4cd6-8dc1-0199d94ca1c9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:34:53.894384 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1088/2195] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Nov 2 07:34:53 2022] DELETE /identity/v3/projects/6672680f45eb4fcba7d107518b7a8796 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:34:53.895941 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628d1d18-2a28-4e22-9169-bd617cd32e76 service neutron] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:34:53.908481 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628d1d18-2a28-4e22-9169-bd617cd32e76 service neutron] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:34:53.958759 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:34:53.975075 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5ecb22-45b1-4569-997b-908d8af1b87c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:34:53.976408 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5ecb22-45b1-4569-997b-908d8af1b87c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:34:53.976783 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5ecb22-45b1-4569-997b-908d8af1b87c admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:34:53.977067 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5ecb22-45b1-4569-997b-908d8af1b87c admin admin] PATH_INFO: `/v3/users/aeeb5d01f71c4f8989969d8f4af8606b` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:34:53.978241 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5ecb22-45b1-4569-997b-908d8af1b87c admin admin] RBAC: Authorizing `identity:delete_user(user_id=aeeb5d01f71c4f8989969d8f4af8606b)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:34:53.980342 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5ecb22-45b1-4569-997b-908d8af1b87c admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:34:54.036957 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-bf5ecb22-45b1-4569-997b-908d8af1b87c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aeeb5d01f71c4f8989969d8f4af8606b'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:34:54.047452 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-bf5ecb22-45b1-4569-997b-908d8af1b87c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aeeb5d01f71c4f8989969d8f4af8606b'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:34:54.048465 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-bf5ecb22-45b1-4569-997b-908d8af1b87c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aeeb5d01f71c4f8989969d8f4af8606b'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:34:54.057118 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-bf5ecb22-45b1-4569-997b-908d8af1b87c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aeeb5d01f71c4f8989969d8f4af8606b'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:34:54.058028 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-bf5ecb22-45b1-4569-997b-908d8af1b87c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aeeb5d01f71c4f8989969d8f4af8606b'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:34:54.084018 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1108/2196] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Nov 2 07:34:53 2022] DELETE /identity/v3/users/aeeb5d01f71c4f8989969d8f4af8606b => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:34:54.410648 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fefd92-5e57-4cd6-8dc1-0199d94ca1c9 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:34:54.412845 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-e4fefd92-5e57-4cd6-8dc1-0199d94ca1c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:34:54.444573 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fefd92-5e57-4cd6-8dc1-0199d94ca1c9 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:34:54.565024 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19d4d4a-2b1c-4fb1-8af3-ebdb70e577be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:34:54.565954 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19d4d4a-2b1c-4fb1-8af3-ebdb70e577be admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:34:54.566444 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19d4d4a-2b1c-4fb1-8af3-ebdb70e577be admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:34:54.566787 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19d4d4a-2b1c-4fb1-8af3-ebdb70e577be admin admin] PATH_INFO: `/v3/projects/2146095387254dd58bf0dc6ed1193957` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:34:54.568151 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19d4d4a-2b1c-4fb1-8af3-ebdb70e577be admin admin] RBAC: Authorizing `identity:delete_project(project_id=2146095387254dd58bf0dc6ed1193957)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:34:54.570765 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19d4d4a-2b1c-4fb1-8af3-ebdb70e577be admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:34:54.631002 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-a19d4d4a-2b1c-4fb1-8af3-ebdb70e577be admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2146095387254dd58bf0dc6ed1193957', 'request_id': 'req-a19d4d4a-2b1c-4fb1-8af3-ebdb70e577be'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:34:54.638898 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-a19d4d4a-2b1c-4fb1-8af3-ebdb70e577be admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2146095387254dd58bf0dc6ed1193957', 'request_id': 'req-a19d4d4a-2b1c-4fb1-8af3-ebdb70e577be'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:34:54.654619 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-a19d4d4a-2b1c-4fb1-8af3-ebdb70e577be admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2146095387254dd58bf0dc6ed1193957', 'request_id': 'req-a19d4d4a-2b1c-4fb1-8af3-ebdb70e577be'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:34:54.670532 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-a19d4d4a-2b1c-4fb1-8af3-ebdb70e577be admin admin] The token cache is being invalidate because project 2146095387254dd58bf0dc6ed1193957 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81501) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:34:54.670831 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-a19d4d4a-2b1c-4fb1-8af3-ebdb70e577be admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:34:54.671765 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-a19d4d4a-2b1c-4fb1-8af3-ebdb70e577be admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:34:54.673400 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1089/2197] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Nov 2 07:34:54 2022] DELETE /identity/v3/projects/2146095387254dd58bf0dc6ed1193957 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:34:54.679734 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db25c8e0-f700-436e-a608-1daedbacde93 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:34:54.680117 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db25c8e0-f700-436e-a608-1daedbacde93 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:34:54.680390 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db25c8e0-f700-436e-a608-1daedbacde93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:34:54.699675 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-db25c8e0-f700-436e-a608-1daedbacde93 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:34:54.700867 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-db25c8e0-f700-436e-a608-1daedbacde93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:34:54.765198 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1109/2198] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:34:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:34:54.775268 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9425d750-7692-495a-b53d-5a30fa1a92a5 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:34:54.775624 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9425d750-7692-495a-b53d-5a30fa1a92a5 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:34:54.776517 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9425d750-7692-495a-b53d-5a30fa1a92a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:34:54.814113 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-9425d750-7692-495a-b53d-5a30fa1a92a5 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:34:54.815393 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-9425d750-7692-495a-b53d-5a30fa1a92a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:34:54.825131 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1090/2199] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:34:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:34:54.829307 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db25c8e0-f700-436e-a608-1daedbacde93 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:34:54.839112 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db25c8e0-f700-436e-a608-1daedbacde93 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:34:54.843541 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c094e49b-6328-4514-a9f6-972567d82399 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:34:54.844137 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c094e49b-6328-4514-a9f6-972567d82399 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:34:54.844412 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c094e49b-6328-4514-a9f6-972567d82399 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:34:54.844681 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c094e49b-6328-4514-a9f6-972567d82399 admin admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:34:54.845317 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c094e49b-6328-4514-a9f6-972567d82399 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:34:54.845612 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c094e49b-6328-4514-a9f6-972567d82399 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:34:54.847261 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c094e49b-6328-4514-a9f6-972567d82399 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:34:54.859972 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1110/2200] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Nov 2 07:34:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:34:54.864371 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9425d750-7692-495a-b53d-5a30fa1a92a5 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:34:54.875620 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9425d750-7692-495a-b53d-5a30fa1a92a5 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:34:54.880594 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec354f3-47e4-4980-91e4-facbfc138e34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:34:54.881211 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec354f3-47e4-4980-91e4-facbfc138e34 admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:34:54.881491 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec354f3-47e4-4980-91e4-facbfc138e34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:34:54.881791 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec354f3-47e4-4980-91e4-facbfc138e34 admin admin] PATH_INFO: `/v3/projects` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:34:54.882522 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec354f3-47e4-4980-91e4-facbfc138e34 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:34:54.884415 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec354f3-47e4-4980-91e4-facbfc138e34 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:34:54.900293 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:34:54.915721 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1091/2201] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Nov 2 07:34:54 2022] POST /identity/v3/projects => generated 385 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:34:54.919989 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c094e49b-6328-4514-a9f6-972567d82399 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:34:54.931270 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c094e49b-6328-4514-a9f6-972567d82399 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:34:54.987723 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:34:54.999458 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d4d85b-7957-4dd7-a6b3-41c9b66ea103 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:34:55.000338 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d4d85b-7957-4dd7-a6b3-41c9b66ea103 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:34:55.000775 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d4d85b-7957-4dd7-a6b3-41c9b66ea103 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:34:55.001252 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d4d85b-7957-4dd7-a6b3-41c9b66ea103 admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:34:55.001889 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d4d85b-7957-4dd7-a6b3-41c9b66ea103 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:34:55.004181 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d4d85b-7957-4dd7-a6b3-41c9b66ea103 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:34:55.025354 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1111/2202] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Nov 2 07:34:54 2022] POST /identity/v3/users => generated 293 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 07:34:55.029890 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec354f3-47e4-4980-91e4-facbfc138e34 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:34:55.040928 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec354f3-47e4-4980-91e4-facbfc138e34 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:34:55.045658 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f937a3e-50ba-4784-b25b-0c6d1c617676 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:34:55.046253 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f937a3e-50ba-4784-b25b-0c6d1c617676 admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:34:55.046563 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f937a3e-50ba-4784-b25b-0c6d1c617676 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:34:55.046840 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f937a3e-50ba-4784-b25b-0c6d1c617676 admin admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:34:55.047475 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f937a3e-50ba-4784-b25b-0c6d1c617676 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:34:55.049069 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f937a3e-50ba-4784-b25b-0c6d1c617676 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:34:55.059270 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1092/2203] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:34:55 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 07:34:55.063183 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d4d85b-7957-4dd7-a6b3-41c9b66ea103 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:34:55.073191 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d4d85b-7957-4dd7-a6b3-41c9b66ea103 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:34:55.077845 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affc8b77-7c12-4db2-a775-917a06e634a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:34:55.078486 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affc8b77-7c12-4db2-a775-917a06e634a1 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:34:55.078767 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affc8b77-7c12-4db2-a775-917a06e634a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:34:55.079039 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affc8b77-7c12-4db2-a775-917a06e634a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:34:55.079673 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affc8b77-7c12-4db2-a775-917a06e634a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:34:55.081243 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affc8b77-7c12-4db2-a775-917a06e634a1 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:34:55.093511 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1112/2204] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:34:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:34:55.097895 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f937a3e-50ba-4784-b25b-0c6d1c617676 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:34:55.109700 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f937a3e-50ba-4784-b25b-0c6d1c617676 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:34:55.114689 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629d8d97-a1c9-4300-9b2d-9371a2f3ad18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:34:55.115283 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629d8d97-a1c9-4300-9b2d-9371a2f3ad18 admin admin] REQUEST_METHOD: `PUT` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:34:55.115565 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629d8d97-a1c9-4300-9b2d-9371a2f3ad18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:34:55.115854 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629d8d97-a1c9-4300-9b2d-9371a2f3ad18 admin admin] PATH_INFO: `/v3/projects/fd68dc56362e426ebd5b2a6f6579b0d4/users/4a1a970f588f4a2a92e49a5964f7927c/roles/b494fedd00f24f0b84a529e8672a156a` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:34:55.134619 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629d8d97-a1c9-4300-9b2d-9371a2f3ad18 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd68dc56362e426ebd5b2a6f6579b0d4, user_id=4a1a970f588f4a2a92e49a5964f7927c, role_id=b494fedd00f24f0b84a529e8672a156a)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:34:55.137384 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629d8d97-a1c9-4300-9b2d-9371a2f3ad18 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:34:55.154751 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1093/2205] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Nov 2 07:34:55 2022] PUT /identity/v3/projects/fd68dc56362e426ebd5b2a6f6579b0d4/users/4a1a970f588f4a2a92e49a5964f7927c/roles/b494fedd00f24f0b84a529e8672a156a => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:34:55.173272 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27f055c-efcd-4daa-b808-da4e1ad36808 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:34:55.173567 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27f055c-efcd-4daa-b808-da4e1ad36808 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:34:55.173957 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27f055c-efcd-4daa-b808-da4e1ad36808 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:34:55.209171 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-e27f055c-efcd-4daa-b808-da4e1ad36808 None None] MFA Rules not processed for user `4a1a970f588f4a2a92e49a5964f7927c`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:34:55.211122 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-e27f055c-efcd-4daa-b808-da4e1ad36808 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:34:55.312837 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1113/2206] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:34:55 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:34:55.313958 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629d8d97-a1c9-4300-9b2d-9371a2f3ad18 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:34:55.412166 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629d8d97-a1c9-4300-9b2d-9371a2f3ad18 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:34:55.421035 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ca4f14-149a-456e-9467-3922166a827c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:34:55.422049 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ca4f14-149a-456e-9467-3922166a827c admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:34:55.422468 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ca4f14-149a-456e-9467-3922166a827c admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:34:55.422843 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ca4f14-149a-456e-9467-3922166a827c admin admin] PATH_INFO: `/v3/projects` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:34:55.423500 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ca4f14-149a-456e-9467-3922166a827c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:34:55.428023 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ca4f14-149a-456e-9467-3922166a827c admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:34:55.472911 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1094/2207] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Nov 2 07:34:55 2022] POST /identity/v3/projects => generated 383 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:34:55.476850 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27f055c-efcd-4daa-b808-da4e1ad36808 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:34:55.487690 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27f055c-efcd-4daa-b808-da4e1ad36808 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:34:55.562810 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97972175-4530-4d48-9491-aa7a3333fd0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:34:55.563424 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97972175-4530-4d48-9491-aa7a3333fd0e admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:34:55.563832 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97972175-4530-4d48-9491-aa7a3333fd0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:34:55.564113 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97972175-4530-4d48-9491-aa7a3333fd0e admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:34:55.566076 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97972175-4530-4d48-9491-aa7a3333fd0e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:34:55.570395 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97972175-4530-4d48-9491-aa7a3333fd0e admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:34:55.602392 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1114/2208] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Nov 2 07:34:55 2022] POST /identity/v3/users => generated 292 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:34:55.604305 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ca4f14-149a-456e-9467-3922166a827c admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:34:55.615269 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ca4f14-149a-456e-9467-3922166a827c admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:34:55.622045 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192c8769-b1de-46b5-a82b-dd680a82c5b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:34:55.622624 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192c8769-b1de-46b5-a82b-dd680a82c5b4 admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:34:55.622906 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192c8769-b1de-46b5-a82b-dd680a82c5b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:34:55.623185 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192c8769-b1de-46b5-a82b-dd680a82c5b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:34:55.625337 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192c8769-b1de-46b5-a82b-dd680a82c5b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:34:55.625752 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192c8769-b1de-46b5-a82b-dd680a82c5b4 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:34:55.640945 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1095/2209] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:34:55 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:34:55.644358 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97972175-4530-4d48-9491-aa7a3333fd0e admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:34:55.669942 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97972175-4530-4d48-9491-aa7a3333fd0e admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:34:55.675493 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477adc9a-8af5-4d15-b30b-882b03150c23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:34:55.676419 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477adc9a-8af5-4d15-b30b-882b03150c23 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:34:55.677355 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477adc9a-8af5-4d15-b30b-882b03150c23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:34:55.677627 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477adc9a-8af5-4d15-b30b-882b03150c23 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:34:55.678330 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477adc9a-8af5-4d15-b30b-882b03150c23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:34:55.680159 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477adc9a-8af5-4d15-b30b-882b03150c23 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:34:55.691174 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1115/2210] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:34:55 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:34:55.695426 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192c8769-b1de-46b5-a82b-dd680a82c5b4 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:34:55.706657 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192c8769-b1de-46b5-a82b-dd680a82c5b4 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:34:55.717318 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b4bf6e-a41b-4175-a06c-10852ea9a0ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:34:55.719988 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b4bf6e-a41b-4175-a06c-10852ea9a0ec admin admin] REQUEST_METHOD: `PUT` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:34:55.721116 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b4bf6e-a41b-4175-a06c-10852ea9a0ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:34:55.721615 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b4bf6e-a41b-4175-a06c-10852ea9a0ec admin admin] PATH_INFO: `/v3/projects/2a73e38ac13e4e90ba4445719682a35b/users/ee44225fb19e46f3b64543c2dd1bc415/roles/b494fedd00f24f0b84a529e8672a156a` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:34:55.754293 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b4bf6e-a41b-4175-a06c-10852ea9a0ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a73e38ac13e4e90ba4445719682a35b, user_id=ee44225fb19e46f3b64543c2dd1bc415, role_id=b494fedd00f24f0b84a529e8672a156a)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:34:55.757551 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b4bf6e-a41b-4175-a06c-10852ea9a0ec admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:34:55.787639 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac51304-0a01-4eea-ba86-edae4ae0952e None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:34:55.787931 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac51304-0a01-4eea-ba86-edae4ae0952e None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:34:55.788517 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac51304-0a01-4eea-ba86-edae4ae0952e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:34:55.791893 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1096/2211] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Nov 2 07:34:55 2022] PUT /identity/v3/projects/2a73e38ac13e4e90ba4445719682a35b/users/ee44225fb19e46f3b64543c2dd1bc415/roles/b494fedd00f24f0b84a529e8672a156a => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 07:34:55.821034 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-bac51304-0a01-4eea-ba86-edae4ae0952e None None] MFA Rules not processed for user `ee44225fb19e46f3b64543c2dd1bc415`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:34:55.822965 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-bac51304-0a01-4eea-ba86-edae4ae0952e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:34:55.862601 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:34:55.911134 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1116/2212] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:34:55 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:34:55.923628 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b4bf6e-a41b-4175-a06c-10852ea9a0ec admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:34:55.999648 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b4bf6e-a41b-4175-a06c-10852ea9a0ec admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:34:56.005060 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feac2bda-9a4a-492f-b4f8-f6abc1f0494a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:34:56.005672 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feac2bda-9a4a-492f-b4f8-f6abc1f0494a admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:34:56.005948 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feac2bda-9a4a-492f-b4f8-f6abc1f0494a admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:34:56.006242 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feac2bda-9a4a-492f-b4f8-f6abc1f0494a admin admin] PATH_INFO: `/v3/projects` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:34:56.007044 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feac2bda-9a4a-492f-b4f8-f6abc1f0494a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:34:56.009128 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feac2bda-9a4a-492f-b4f8-f6abc1f0494a admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:34:56.026295 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:34:56.044449 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1097/2213] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Nov 2 07:34:55 2022] POST /identity/v3/projects => generated 383 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:34:56.048755 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac51304-0a01-4eea-ba86-edae4ae0952e None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:34:56.062262 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac51304-0a01-4eea-ba86-edae4ae0952e None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:34:56.082728 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feac2bda-9a4a-492f-b4f8-f6abc1f0494a admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:34:56.087810 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-feac2bda-9a4a-492f-b4f8-f6abc1f0494a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:34:56.160038 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63eb2985-cc65-42df-850f-9cacf3c0c2dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:34:56.160672 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63eb2985-cc65-42df-850f-9cacf3c0c2dc admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:34:56.160948 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63eb2985-cc65-42df-850f-9cacf3c0c2dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:34:56.161217 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63eb2985-cc65-42df-850f-9cacf3c0c2dc admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:34:56.161859 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63eb2985-cc65-42df-850f-9cacf3c0c2dc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:34:56.164895 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63eb2985-cc65-42df-850f-9cacf3c0c2dc admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:34:56.202023 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1117/2214] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Nov 2 07:34:56 2022] POST /identity/v3/users => generated 298 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:34:56.204294 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63eb2985-cc65-42df-850f-9cacf3c0c2dc admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:34:56.209094 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feac2bda-9a4a-492f-b4f8-f6abc1f0494a admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:34:56.214716 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63eb2985-cc65-42df-850f-9cacf3c0c2dc admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:34:56.217021 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7806625-9654-496d-8259-72e29390830e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e48cba3f804cf08b2477f01ae95179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:34:56.217862 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7806625-9654-496d-8259-72e29390830e service nova] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:34:56.218297 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7806625-9654-496d-8259-72e29390830e service nova] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:34:56.218766 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7806625-9654-496d-8259-72e29390830e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:34:56.220148 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bf3c48-8b29-42c1-9230-366cd2ba950b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:34:56.220834 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bf3c48-8b29-42c1-9230-366cd2ba950b admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:34:56.221176 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bf3c48-8b29-42c1-9230-366cd2ba950b admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:34:56.221517 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bf3c48-8b29-42c1-9230-366cd2ba950b admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:34:56.222240 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-c7806625-9654-496d-8259-72e29390830e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:34:56.222429 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bf3c48-8b29-42c1-9230-366cd2ba950b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:34:56.224436 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bf3c48-8b29-42c1-9230-366cd2ba950b admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:34:56.238517 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7806625-9654-496d-8259-72e29390830e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:34:56.241252 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7806625-9654-496d-8259-72e29390830e service nova] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:34:56.246577 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1118/2215] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:34:56 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:34:56.251607 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bf3c48-8b29-42c1-9230-366cd2ba950b admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:34:56.258720 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1098/2216] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:34:56 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:34:56.263101 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bf3c48-8b29-42c1-9230-366cd2ba950b admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:34:56.268694 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8614ba13-946c-438f-94ae-f89ab5cd9692 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:34:56.269436 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8614ba13-946c-438f-94ae-f89ab5cd9692 admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:34:56.269718 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8614ba13-946c-438f-94ae-f89ab5cd9692 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:34:56.269989 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8614ba13-946c-438f-94ae-f89ab5cd9692 admin admin] PATH_INFO: `/v3/projects/f648f532ff2d478092e2864409dcb3dc/users/52343f319a614c08b3a48876a52b0538/roles/c4bf019fc6c44c71a6c984f6d0bab75f` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:34:56.289942 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8614ba13-946c-438f-94ae-f89ab5cd9692 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f648f532ff2d478092e2864409dcb3dc, user_id=52343f319a614c08b3a48876a52b0538, role_id=c4bf019fc6c44c71a6c984f6d0bab75f)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:34:56.292885 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8614ba13-946c-438f-94ae-f89ab5cd9692 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:34:56.316437 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1119/2217] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Nov 2 07:34:56 2022] PUT /identity/v3/projects/f648f532ff2d478092e2864409dcb3dc/users/52343f319a614c08b3a48876a52b0538/roles/c4bf019fc6c44c71a6c984f6d0bab75f => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:34:56.318244 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7806625-9654-496d-8259-72e29390830e service nova] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:34:56.331194 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7806625-9654-496d-8259-72e29390830e service nova] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:34:56.398192 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff8666e-cfa3-4f73-9db3-2dcc7d6fdab9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:34:56.398929 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff8666e-cfa3-4f73-9db3-2dcc7d6fdab9 admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:34:56.399216 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff8666e-cfa3-4f73-9db3-2dcc7d6fdab9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:34:56.399488 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff8666e-cfa3-4f73-9db3-2dcc7d6fdab9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:34:56.400131 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff8666e-cfa3-4f73-9db3-2dcc7d6fdab9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:34:56.401730 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff8666e-cfa3-4f73-9db3-2dcc7d6fdab9 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:34:56.412423 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1099/2218] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:34:56 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:34:56.416757 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8614ba13-946c-438f-94ae-f89ab5cd9692 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:34:56.427223 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8614ba13-946c-438f-94ae-f89ab5cd9692 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:34:56.432433 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a369e67c-5377-4316-8122-e35883779ea1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:34:56.433093 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a369e67c-5377-4316-8122-e35883779ea1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:34:56.433372 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a369e67c-5377-4316-8122-e35883779ea1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:34:56.433641 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a369e67c-5377-4316-8122-e35883779ea1 admin admin] PATH_INFO: `/v3/projects/f648f532ff2d478092e2864409dcb3dc/users/52343f319a614c08b3a48876a52b0538/roles/c4bf019fc6c44c71a6c984f6d0bab75f` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:34:56.435560 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a369e67c-5377-4316-8122-e35883779ea1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f648f532ff2d478092e2864409dcb3dc, user_id=52343f319a614c08b3a48876a52b0538, role_id=c4bf019fc6c44c71a6c984f6d0bab75f)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:34:56.438200 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a369e67c-5377-4316-8122-e35883779ea1 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:34:56.449329 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1120/2219] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Nov 2 07:34:56 2022] PUT /identity/v3/projects/f648f532ff2d478092e2864409dcb3dc/users/52343f319a614c08b3a48876a52b0538/roles/c4bf019fc6c44c71a6c984f6d0bab75f => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:34:56.461203 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffd4f2f-6db5-4872-9b55-aaf87127f20f None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:34:56.461645 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffd4f2f-6db5-4872-9b55-aaf87127f20f None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:34:56.461928 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffd4f2f-6db5-4872-9b55-aaf87127f20f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:34:56.502150 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-bffd4f2f-6db5-4872-9b55-aaf87127f20f None None] MFA Rules not processed for user `52343f319a614c08b3a48876a52b0538`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:34:56.503885 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-bffd4f2f-6db5-4872-9b55-aaf87127f20f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:34:56.580855 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1100/2220] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:34:56 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:34:56.592982 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a369e67c-5377-4316-8122-e35883779ea1 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:34:56.594698 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-a369e67c-5377-4316-8122-e35883779ea1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:34:56.674104 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a369e67c-5377-4316-8122-e35883779ea1 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:34:56.678275 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c931624a-4ea2-489d-9717-bdb39de9af62 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43bb326fb82e4d2c83fb2eaf91860dc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:34:56.679544 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c931624a-4ea2-489d-9717-bdb39de9af62 service cinder] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:34:56.680014 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c931624a-4ea2-489d-9717-bdb39de9af62 service cinder] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:34:56.680498 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c931624a-4ea2-489d-9717-bdb39de9af62 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:34:56.720726 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:34:56.737440 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40b722f-220d-4f9a-990b-6557c4d983cf None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:34:56.738055 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40b722f-220d-4f9a-990b-6557c4d983cf None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:34:56.738564 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40b722f-220d-4f9a-990b-6557c4d983cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:34:56.746708 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:34:56.766300 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c931624a-4ea2-489d-9717-bdb39de9af62 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:34:56.767773 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c931624a-4ea2-489d-9717-bdb39de9af62 service cinder] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:34:56.780392 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1121/2221] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:34:56 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:34:56.800669 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-e40b722f-220d-4f9a-990b-6557c4d983cf None None] MFA Rules not processed for user `ff4736f3d50746b0bf5d6a917b2ca0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:34:56.803150 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-e40b722f-220d-4f9a-990b-6557c4d983cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:34:56.879031 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1101/2222] 10.222.0.18 () {58 vars in 1009 bytes} [Wed Nov 2 07:34:56 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:34:56.893540 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c931624a-4ea2-489d-9717-bdb39de9af62 service cinder] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:34:56.895448 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-c931624a-4ea2-489d-9717-bdb39de9af62 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:34:56.987354 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c931624a-4ea2-489d-9717-bdb39de9af62 service cinder] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:34:56.994913 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1ee478-8edc-49be-b0a1-8d7ae5ccdb5d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9116f47889f4db39e0203d6cdae1593', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:34:56.995556 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1ee478-8edc-49be-b0a1-8d7ae5ccdb5d service swift] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:34:56.995936 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1ee478-8edc-49be-b0a1-8d7ae5ccdb5d service swift] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:34:56.996102 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1ee478-8edc-49be-b0a1-8d7ae5ccdb5d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:34:57.006158 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1ee478-8edc-49be-b0a1-8d7ae5ccdb5d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:34:57.009182 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1ee478-8edc-49be-b0a1-8d7ae5ccdb5d service swift] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:34:57.018015 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1122/2223] 10.222.0.18 () {60 vars in 1384 bytes} [Wed Nov 2 07:34:56 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Nov 02 07:34:58.542256 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40b722f-220d-4f9a-990b-6557c4d983cf None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:34:58.544685 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-e40b722f-220d-4f9a-990b-6557c4d983cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:34:58.557584 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40b722f-220d-4f9a-990b-6557c4d983cf None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:34:58.561430 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a2ee8f-452e-44c2-8b9d-a8392ae168e1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43bb326fb82e4d2c83fb2eaf91860dc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:34:58.562121 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a2ee8f-452e-44c2-8b9d-a8392ae168e1 service cinder] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:34:58.562501 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a2ee8f-452e-44c2-8b9d-a8392ae168e1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:34:58.562849 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a2ee8f-452e-44c2-8b9d-a8392ae168e1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:34:58.640521 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a2ee8f-452e-44c2-8b9d-a8392ae168e1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:34:58.642172 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a2ee8f-452e-44c2-8b9d-a8392ae168e1 service cinder] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:34:58.661655 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1102/2224] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:34:58 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:34:59.333929 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1ee478-8edc-49be-b0a1-8d7ae5ccdb5d service swift] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:34:59.349222 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1ee478-8edc-49be-b0a1-8d7ae5ccdb5d service swift] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:34:59.356190 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e6d023-2688-4b7b-a25f-7266d5adb2f4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43bb326fb82e4d2c83fb2eaf91860dc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:34:59.356813 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e6d023-2688-4b7b-a25f-7266d5adb2f4 service cinder] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:34:59.357095 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e6d023-2688-4b7b-a25f-7266d5adb2f4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:34:59.357370 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e6d023-2688-4b7b-a25f-7266d5adb2f4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:34:59.392199 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e6d023-2688-4b7b-a25f-7266d5adb2f4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:34:59.393802 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e6d023-2688-4b7b-a25f-7266d5adb2f4 service cinder] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:34:59.420924 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1123/2225] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:34:59 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:35:00.361119 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a2ee8f-452e-44c2-8b9d-a8392ae168e1 service cinder] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:35:00.362782 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-d4a2ee8f-452e-44c2-8b9d-a8392ae168e1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:35:00.466560 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a2ee8f-452e-44c2-8b9d-a8392ae168e1 service cinder] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:35:00.472981 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2aecb25-1923-4243-8995-a32fb032fc2c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e48cba3f804cf08b2477f01ae95179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:35:00.473651 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2aecb25-1923-4243-8995-a32fb032fc2c service nova] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:00.474044 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2aecb25-1923-4243-8995-a32fb032fc2c service nova] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:00.474321 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2aecb25-1923-4243-8995-a32fb032fc2c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:00.476718 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-e2aecb25-1923-4243-8995-a32fb032fc2c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:35:00.489812 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2aecb25-1923-4243-8995-a32fb032fc2c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:35:00.491903 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2aecb25-1923-4243-8995-a32fb032fc2c service nova] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:35:00.512296 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1103/2226] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:35:00 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:35:03.638197 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e6d023-2688-4b7b-a25f-7266d5adb2f4 service cinder] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:35:03.718124 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e6d023-2688-4b7b-a25f-7266d5adb2f4 service cinder] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:35:03.722797 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824989a7-528e-4fae-a5dc-29fcabf7c03b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:35:03.723622 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824989a7-528e-4fae-a5dc-29fcabf7c03b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:03.723963 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824989a7-528e-4fae-a5dc-29fcabf7c03b admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:03.724326 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824989a7-528e-4fae-a5dc-29fcabf7c03b admin admin] PATH_INFO: `/v3/users/4a1a970f588f4a2a92e49a5964f7927c` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:03.725492 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824989a7-528e-4fae-a5dc-29fcabf7c03b admin admin] RBAC: Authorizing `identity:delete_user(user_id=4a1a970f588f4a2a92e49a5964f7927c)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:35:03.728007 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824989a7-528e-4fae-a5dc-29fcabf7c03b admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:35:03.777989 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:35:03.781213 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-824989a7-528e-4fae-a5dc-29fcabf7c03b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4a1a970f588f4a2a92e49a5964f7927c'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:35:03.791569 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-824989a7-528e-4fae-a5dc-29fcabf7c03b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4a1a970f588f4a2a92e49a5964f7927c'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:35:03.793260 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-824989a7-528e-4fae-a5dc-29fcabf7c03b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4a1a970f588f4a2a92e49a5964f7927c'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:35:03.801682 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-824989a7-528e-4fae-a5dc-29fcabf7c03b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4a1a970f588f4a2a92e49a5964f7927c'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:35:03.802584 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-824989a7-528e-4fae-a5dc-29fcabf7c03b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4a1a970f588f4a2a92e49a5964f7927c'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:35:03.820673 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1124/2227] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Nov 2 07:35:03 2022] DELETE /identity/v3/users/4a1a970f588f4a2a92e49a5964f7927c => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:35:03.825439 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d75b801-dfc6-4471-8681-a85eec12ca57 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:03.825954 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d75b801-dfc6-4471-8681-a85eec12ca57 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:03.825954 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d75b801-dfc6-4471-8681-a85eec12ca57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:03.845419 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-0d75b801-dfc6-4471-8681-a85eec12ca57 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:35:03.846635 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-0d75b801-dfc6-4471-8681-a85eec12ca57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:35:03.872949 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:35:03.909067 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1104/2228] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:35:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:35:03.918569 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824989a7-528e-4fae-a5dc-29fcabf7c03b admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:35:03.919946 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-824989a7-528e-4fae-a5dc-29fcabf7c03b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:35:03.994585 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824989a7-528e-4fae-a5dc-29fcabf7c03b admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:35:03.998779 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dec3700-0ec5-44b4-9e10-48b7565f9772 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3ec6ecb85f94ef59762877d9ea33ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:35:03.999541 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dec3700-0ec5-44b4-9e10-48b7565f9772 service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:03.999899 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dec3700-0ec5-44b4-9e10-48b7565f9772 service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:04.000171 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dec3700-0ec5-44b4-9e10-48b7565f9772 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:04.013668 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dec3700-0ec5-44b4-9e10-48b7565f9772 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:35:04.015138 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dec3700-0ec5-44b4-9e10-48b7565f9772 service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:35:04.025155 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1125/2229] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:35:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:35:04.337421 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d75b801-dfc6-4471-8681-a85eec12ca57 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:35:04.338979 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-0d75b801-dfc6-4471-8681-a85eec12ca57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:35:04.364965 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d75b801-dfc6-4471-8681-a85eec12ca57 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:35:04.381200 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e921d54-70ba-4913-b19c-71b4a73959a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:35:04.382912 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e921d54-70ba-4913-b19c-71b4a73959a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:04.383353 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e921d54-70ba-4913-b19c-71b4a73959a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:04.383702 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e921d54-70ba-4913-b19c-71b4a73959a7 admin admin] PATH_INFO: `/v3/projects/fd68dc56362e426ebd5b2a6f6579b0d4` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:04.386702 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e921d54-70ba-4913-b19c-71b4a73959a7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fd68dc56362e426ebd5b2a6f6579b0d4)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:35:04.388731 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e921d54-70ba-4913-b19c-71b4a73959a7 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:35:04.482308 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-9e921d54-70ba-4913-b19c-71b4a73959a7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fd68dc56362e426ebd5b2a6f6579b0d4', 'request_id': 'req-9e921d54-70ba-4913-b19c-71b4a73959a7'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:35:04.493553 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-9e921d54-70ba-4913-b19c-71b4a73959a7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fd68dc56362e426ebd5b2a6f6579b0d4', 'request_id': 'req-9e921d54-70ba-4913-b19c-71b4a73959a7'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:35:04.506035 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-9e921d54-70ba-4913-b19c-71b4a73959a7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fd68dc56362e426ebd5b2a6f6579b0d4', 'request_id': 'req-9e921d54-70ba-4913-b19c-71b4a73959a7'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:35:04.537180 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-9e921d54-70ba-4913-b19c-71b4a73959a7 admin admin] The token cache is being invalidate because project fd68dc56362e426ebd5b2a6f6579b0d4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81501) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:35:04.537729 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-9e921d54-70ba-4913-b19c-71b4a73959a7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:35:04.539479 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-9e921d54-70ba-4913-b19c-71b4a73959a7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:35:04.541163 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1105/2230] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Nov 2 07:35:04 2022] DELETE /identity/v3/projects/fd68dc56362e426ebd5b2a6f6579b0d4 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:35:04.545473 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dec3700-0ec5-44b4-9e10-48b7565f9772 service neutron] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:35:04.559501 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dec3700-0ec5-44b4-9e10-48b7565f9772 service neutron] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:35:04.645023 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7045686b-4596-40eb-822b-5653e9ea7e55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:35:04.645767 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7045686b-4596-40eb-822b-5653e9ea7e55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:04.646141 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7045686b-4596-40eb-822b-5653e9ea7e55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:04.646534 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7045686b-4596-40eb-822b-5653e9ea7e55 admin admin] PATH_INFO: `/v3/users/ee44225fb19e46f3b64543c2dd1bc415` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:04.648050 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7045686b-4596-40eb-822b-5653e9ea7e55 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ee44225fb19e46f3b64543c2dd1bc415)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:35:04.650227 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7045686b-4596-40eb-822b-5653e9ea7e55 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:35:04.697445 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-7045686b-4596-40eb-822b-5653e9ea7e55 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ee44225fb19e46f3b64543c2dd1bc415'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:35:04.704257 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-7045686b-4596-40eb-822b-5653e9ea7e55 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ee44225fb19e46f3b64543c2dd1bc415'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:35:04.704894 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-7045686b-4596-40eb-822b-5653e9ea7e55 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ee44225fb19e46f3b64543c2dd1bc415'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:35:04.714015 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-7045686b-4596-40eb-822b-5653e9ea7e55 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ee44225fb19e46f3b64543c2dd1bc415'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:35:04.714778 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-7045686b-4596-40eb-822b-5653e9ea7e55 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ee44225fb19e46f3b64543c2dd1bc415'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:35:04.721614 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:35:04.747716 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1126/2231] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Nov 2 07:35:04 2022] DELETE /identity/v3/users/ee44225fb19e46f3b64543c2dd1bc415 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:35:05.123544 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e921d54-70ba-4913-b19c-71b4a73959a7 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:35:05.125062 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-9e921d54-70ba-4913-b19c-71b4a73959a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:35:05.159100 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e921d54-70ba-4913-b19c-71b4a73959a7 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:35:05.245988 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081cc4c2-e829-4c08-a05c-0ad30f6d941c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:35:05.246670 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081cc4c2-e829-4c08-a05c-0ad30f6d941c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:05.246951 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081cc4c2-e829-4c08-a05c-0ad30f6d941c admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:05.247224 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081cc4c2-e829-4c08-a05c-0ad30f6d941c admin admin] PATH_INFO: `/v3/projects/2a73e38ac13e4e90ba4445719682a35b` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:05.248362 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081cc4c2-e829-4c08-a05c-0ad30f6d941c admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a73e38ac13e4e90ba4445719682a35b)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:35:05.250298 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081cc4c2-e829-4c08-a05c-0ad30f6d941c admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:35:05.298971 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-081cc4c2-e829-4c08-a05c-0ad30f6d941c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a73e38ac13e4e90ba4445719682a35b', 'request_id': 'req-081cc4c2-e829-4c08-a05c-0ad30f6d941c'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:35:05.305457 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-081cc4c2-e829-4c08-a05c-0ad30f6d941c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a73e38ac13e4e90ba4445719682a35b', 'request_id': 'req-081cc4c2-e829-4c08-a05c-0ad30f6d941c'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:35:05.307962 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:35:05.313462 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-081cc4c2-e829-4c08-a05c-0ad30f6d941c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a73e38ac13e4e90ba4445719682a35b', 'request_id': 'req-081cc4c2-e829-4c08-a05c-0ad30f6d941c'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:35:05.325563 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-081cc4c2-e829-4c08-a05c-0ad30f6d941c admin admin] The token cache is being invalidate because project 2a73e38ac13e4e90ba4445719682a35b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81501) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:35:05.325844 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-081cc4c2-e829-4c08-a05c-0ad30f6d941c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:35:05.326500 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-081cc4c2-e829-4c08-a05c-0ad30f6d941c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:35:05.327996 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1106/2232] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Nov 2 07:35:05 2022] DELETE /identity/v3/projects/2a73e38ac13e4e90ba4445719682a35b => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:35:05.331976 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7045686b-4596-40eb-822b-5653e9ea7e55 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:35:05.333414 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-7045686b-4596-40eb-822b-5653e9ea7e55 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:35:05.346715 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7045686b-4596-40eb-822b-5653e9ea7e55 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:35:05.416595 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7aafe5c-9dc1-4b06-9833-da91a884730c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:35:05.417235 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7aafe5c-9dc1-4b06-9833-da91a884730c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:05.417513 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7aafe5c-9dc1-4b06-9833-da91a884730c admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:05.417777 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7aafe5c-9dc1-4b06-9833-da91a884730c admin admin] PATH_INFO: `/v3/users/52343f319a614c08b3a48876a52b0538` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:05.418987 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7aafe5c-9dc1-4b06-9833-da91a884730c admin admin] RBAC: Authorizing `identity:delete_user(user_id=52343f319a614c08b3a48876a52b0538)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:35:05.421036 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7aafe5c-9dc1-4b06-9833-da91a884730c admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:35:05.483736 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-c7aafe5c-9dc1-4b06-9833-da91a884730c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '52343f319a614c08b3a48876a52b0538'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:35:05.491070 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-c7aafe5c-9dc1-4b06-9833-da91a884730c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '52343f319a614c08b3a48876a52b0538'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:35:05.492335 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-c7aafe5c-9dc1-4b06-9833-da91a884730c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '52343f319a614c08b3a48876a52b0538'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:35:05.499660 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-c7aafe5c-9dc1-4b06-9833-da91a884730c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '52343f319a614c08b3a48876a52b0538'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:35:05.500363 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-c7aafe5c-9dc1-4b06-9833-da91a884730c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '52343f319a614c08b3a48876a52b0538'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:35:05.518593 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1127/2233] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Nov 2 07:35:05 2022] DELETE /identity/v3/users/52343f319a614c08b3a48876a52b0538 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:35:05.866433 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081cc4c2-e829-4c08-a05c-0ad30f6d941c admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:35:05.867913 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-081cc4c2-e829-4c08-a05c-0ad30f6d941c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:35:05.885265 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081cc4c2-e829-4c08-a05c-0ad30f6d941c admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:35:05.950271 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94aead36-d8d4-4e4b-bfc1-02376629f5d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:35:05.951273 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94aead36-d8d4-4e4b-bfc1-02376629f5d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:05.951559 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94aead36-d8d4-4e4b-bfc1-02376629f5d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:05.951846 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94aead36-d8d4-4e4b-bfc1-02376629f5d8 admin admin] PATH_INFO: `/v3/projects/f648f532ff2d478092e2864409dcb3dc` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:05.953021 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94aead36-d8d4-4e4b-bfc1-02376629f5d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f648f532ff2d478092e2864409dcb3dc)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:35:05.954904 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94aead36-d8d4-4e4b-bfc1-02376629f5d8 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:35:06.015599 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-94aead36-d8d4-4e4b-bfc1-02376629f5d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f648f532ff2d478092e2864409dcb3dc', 'request_id': 'req-94aead36-d8d4-4e4b-bfc1-02376629f5d8'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:35:06.022459 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-94aead36-d8d4-4e4b-bfc1-02376629f5d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f648f532ff2d478092e2864409dcb3dc', 'request_id': 'req-94aead36-d8d4-4e4b-bfc1-02376629f5d8'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:35:06.030913 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-94aead36-d8d4-4e4b-bfc1-02376629f5d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f648f532ff2d478092e2864409dcb3dc', 'request_id': 'req-94aead36-d8d4-4e4b-bfc1-02376629f5d8'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:35:06.042444 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-94aead36-d8d4-4e4b-bfc1-02376629f5d8 admin admin] The token cache is being invalidate because project f648f532ff2d478092e2864409dcb3dc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81501) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:35:06.042739 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-94aead36-d8d4-4e4b-bfc1-02376629f5d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:35:06.043397 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-94aead36-d8d4-4e4b-bfc1-02376629f5d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:35:06.048202 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1107/2234] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Nov 2 07:35:05 2022] DELETE /identity/v3/projects/f648f532ff2d478092e2864409dcb3dc => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:35:06.049823 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4f0c45-6261-4377-87ec-3a789ce2f0f8 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:06.050335 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4f0c45-6261-4377-87ec-3a789ce2f0f8 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:06.050453 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4f0c45-6261-4377-87ec-3a789ce2f0f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:06.071835 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-be4f0c45-6261-4377-87ec-3a789ce2f0f8 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:35:06.073215 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-be4f0c45-6261-4377-87ec-3a789ce2f0f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:35:06.134048 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1128/2235] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:35:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:35:06.147712 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79644aa-0aa3-4920-a8fe-b9f01c91cd03 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:06.148309 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79644aa-0aa3-4920-a8fe-b9f01c91cd03 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:06.148530 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79644aa-0aa3-4920-a8fe-b9f01c91cd03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:06.195370 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-e79644aa-0aa3-4920-a8fe-b9f01c91cd03 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:35:06.196705 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-e79644aa-0aa3-4920-a8fe-b9f01c91cd03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:35:06.205636 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1108/2236] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:35:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:35:06.211964 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4f0c45-6261-4377-87ec-3a789ce2f0f8 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:35:06.217808 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:35:06.224284 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4f0c45-6261-4377-87ec-3a789ce2f0f8 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:35:06.236480 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901206e0-8c36-412b-95d3-9098a4852e13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:35:06.237614 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901206e0-8c36-412b-95d3-9098a4852e13 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:06.237955 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901206e0-8c36-412b-95d3-9098a4852e13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:06.238289 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901206e0-8c36-412b-95d3-9098a4852e13 admin admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:06.239278 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901206e0-8c36-412b-95d3-9098a4852e13 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:35:06.240250 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901206e0-8c36-412b-95d3-9098a4852e13 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:35:06.242425 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901206e0-8c36-412b-95d3-9098a4852e13 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:35:06.256960 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1129/2237] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Nov 2 07:35:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:35:06.261763 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79644aa-0aa3-4920-a8fe-b9f01c91cd03 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:35:06.271355 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79644aa-0aa3-4920-a8fe-b9f01c91cd03 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:35:06.276700 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d107e0a7-1de6-4c38-8431-c774b52ffa36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:35:06.277376 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d107e0a7-1de6-4c38-8431-c774b52ffa36 admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:06.277766 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d107e0a7-1de6-4c38-8431-c774b52ffa36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:06.278111 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d107e0a7-1de6-4c38-8431-c774b52ffa36 admin admin] PATH_INFO: `/v3/projects` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:06.278860 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d107e0a7-1de6-4c38-8431-c774b52ffa36 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:35:06.280845 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d107e0a7-1de6-4c38-8431-c774b52ffa36 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:35:06.309451 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1109/2238] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Nov 2 07:35:06 2022] POST /identity/v3/projects => generated 369 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:35:06.314242 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901206e0-8c36-412b-95d3-9098a4852e13 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:35:06.325389 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901206e0-8c36-412b-95d3-9098a4852e13 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:35:06.384616 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12634e9c-d3e8-4e49-ae4f-2878208e0c10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:35:06.385228 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12634e9c-d3e8-4e49-ae4f-2878208e0c10 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:06.385511 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12634e9c-d3e8-4e49-ae4f-2878208e0c10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:06.385782 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12634e9c-d3e8-4e49-ae4f-2878208e0c10 admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:06.386451 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12634e9c-d3e8-4e49-ae4f-2878208e0c10 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:35:06.388354 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12634e9c-d3e8-4e49-ae4f-2878208e0c10 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:35:06.409475 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1130/2239] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Nov 2 07:35:06 2022] POST /identity/v3/users => generated 285 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:35:06.414161 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d107e0a7-1de6-4c38-8431-c774b52ffa36 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:35:06.424528 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d107e0a7-1de6-4c38-8431-c774b52ffa36 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:35:06.429426 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc30974b-2491-4da3-afed-1181dedd9c57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:35:06.430022 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc30974b-2491-4da3-afed-1181dedd9c57 admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:06.430304 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc30974b-2491-4da3-afed-1181dedd9c57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:06.430630 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc30974b-2491-4da3-afed-1181dedd9c57 admin admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:06.431290 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc30974b-2491-4da3-afed-1181dedd9c57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:35:06.432976 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc30974b-2491-4da3-afed-1181dedd9c57 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:35:06.442940 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1110/2240] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:35:06 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:35:06.446820 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12634e9c-d3e8-4e49-ae4f-2878208e0c10 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:35:06.457625 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12634e9c-d3e8-4e49-ae4f-2878208e0c10 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:35:06.463243 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2faf0746-5c32-47d0-b3bb-8858a322b629 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:35:06.463833 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faf0746-5c32-47d0-b3bb-8858a322b629 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:06.464115 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faf0746-5c32-47d0-b3bb-8858a322b629 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:06.464396 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faf0746-5c32-47d0-b3bb-8858a322b629 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:06.465027 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2faf0746-5c32-47d0-b3bb-8858a322b629 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:35:06.466638 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2faf0746-5c32-47d0-b3bb-8858a322b629 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:35:06.478347 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1131/2241] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:35:06 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:35:06.484146 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc30974b-2491-4da3-afed-1181dedd9c57 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:35:06.499628 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc30974b-2491-4da3-afed-1181dedd9c57 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:35:06.507677 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2a21f1-b2c9-4a06-9c78-00eaddec4366 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:35:06.508445 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2a21f1-b2c9-4a06-9c78-00eaddec4366 admin admin] REQUEST_METHOD: `PUT` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:06.508732 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2a21f1-b2c9-4a06-9c78-00eaddec4366 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:06.509076 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2a21f1-b2c9-4a06-9c78-00eaddec4366 admin admin] PATH_INFO: `/v3/projects/cfd4f4705f464cd2ab4f3c35747b169f/users/76946b04f98b47749cd2d8da5a2dae45/roles/b494fedd00f24f0b84a529e8672a156a` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:06.533175 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2a21f1-b2c9-4a06-9c78-00eaddec4366 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cfd4f4705f464cd2ab4f3c35747b169f, user_id=76946b04f98b47749cd2d8da5a2dae45, role_id=b494fedd00f24f0b84a529e8672a156a)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:35:06.535887 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2a21f1-b2c9-4a06-9c78-00eaddec4366 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:35:06.557780 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1111/2242] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Nov 2 07:35:06 2022] PUT /identity/v3/projects/cfd4f4705f464cd2ab4f3c35747b169f/users/76946b04f98b47749cd2d8da5a2dae45/roles/b494fedd00f24f0b84a529e8672a156a => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:35:06.630325 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366c94fd-4972-4a54-9caf-0aea360413c5 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:06.630646 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366c94fd-4972-4a54-9caf-0aea360413c5 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:06.630920 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366c94fd-4972-4a54-9caf-0aea360413c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:06.660569 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-366c94fd-4972-4a54-9caf-0aea360413c5 None None] MFA Rules not processed for user `76946b04f98b47749cd2d8da5a2dae45`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:35:06.661777 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-366c94fd-4972-4a54-9caf-0aea360413c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:35:06.736080 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1132/2243] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:35:06 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:35:06.749794 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2a21f1-b2c9-4a06-9c78-00eaddec4366 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:35:06.754080 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-9c2a21f1-b2c9-4a06-9c78-00eaddec4366 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:35:06.772139 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:DELETE=10,keystone:SELECT=87,keystone:INSERT=3 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:35:06.836997 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2a21f1-b2c9-4a06-9c78-00eaddec4366 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:35:06.842293 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9e47f9-188d-407b-83b7-9cf9ef488a06 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5353f2af0f44d668e2a269647aa1dc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:35:06.842950 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9e47f9-188d-407b-83b7-9cf9ef488a06 service glance] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:06.843216 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9e47f9-188d-407b-83b7-9cf9ef488a06 service glance] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:06.843498 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9e47f9-188d-407b-83b7-9cf9ef488a06 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:06.853863 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9e47f9-188d-407b-83b7-9cf9ef488a06 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:35:06.855538 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9e47f9-188d-407b-83b7-9cf9ef488a06 service glance] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:35:06.865595 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1112/2244] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:35:06 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:35:13.437418 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d047ef-5794-4ea6-bec8-bada14a1cb07 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:13.437975 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d047ef-5794-4ea6-bec8-bada14a1cb07 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:13.437975 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d047ef-5794-4ea6-bec8-bada14a1cb07 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:13.439311 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1133/2245] 10.222.0.18 () {54 vars in 921 bytes} [Wed Nov 2 07:35:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:35:13.444404 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be2fc6d-ac3c-4606-8215-8c273f11c43d None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:13.445458 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be2fc6d-ac3c-4606-8215-8c273f11c43d None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:13.445458 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be2fc6d-ac3c-4606-8215-8c273f11c43d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:13.469610 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-5be2fc6d-ac3c-4606-8215-8c273f11c43d None None] MFA Rules not processed for user `69e48cba3f804cf08b2477f01ae95179`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:35:13.470945 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-5be2fc6d-ac3c-4606-8215-8c273f11c43d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:35:13.556933 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1113/2246] 10.222.0.18 () {58 vars in 1018 bytes} [Wed Nov 2 07:35:13 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:35:13.568705 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d047ef-5794-4ea6-bec8-bada14a1cb07 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:35:13.571373 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-98d047ef-5794-4ea6-bec8-bada14a1cb07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:35:13.633911 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d047ef-5794-4ea6-bec8-bada14a1cb07 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:35:13.637643 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ade1319-505a-41a1-bc7b-9865826bd3cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3ec6ecb85f94ef59762877d9ea33ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:35:13.638272 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ade1319-505a-41a1-bc7b-9865826bd3cb service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:13.638589 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ade1319-505a-41a1-bc7b-9865826bd3cb service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:13.638866 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ade1319-505a-41a1-bc7b-9865826bd3cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:13.657415 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ade1319-505a-41a1-bc7b-9865826bd3cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:35:13.658988 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ade1319-505a-41a1-bc7b-9865826bd3cb service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:35:13.665154 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:35:13.678771 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1134/2247] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:35:13 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:35:13.785814 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be2fc6d-ac3c-4606-8215-8c273f11c43d None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:35:13.787336 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-5be2fc6d-ac3c-4606-8215-8c273f11c43d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:35:13.807082 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be2fc6d-ac3c-4606-8215-8c273f11c43d None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:35:13.814283 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495f5c6c-fce9-4c85-868d-ea2844d0c9d2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e48cba3f804cf08b2477f01ae95179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:35:13.815001 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495f5c6c-fce9-4c85-868d-ea2844d0c9d2 service nova] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:13.815352 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495f5c6c-fce9-4c85-868d-ea2844d0c9d2 service nova] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:13.816004 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495f5c6c-fce9-4c85-868d-ea2844d0c9d2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:13.818026 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-495f5c6c-fce9-4c85-868d-ea2844d0c9d2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:35:13.831961 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495f5c6c-fce9-4c85-868d-ea2844d0c9d2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:35:13.833684 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495f5c6c-fce9-4c85-868d-ea2844d0c9d2 service nova] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:35:13.851086 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1114/2248] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:35:13 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:35:14.830787 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f581aab3-5a19-46e6-82d6-5fcef92b902c None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:14.831299 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f581aab3-5a19-46e6-82d6-5fcef92b902c None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:14.831392 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f581aab3-5a19-46e6-82d6-5fcef92b902c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:14.870567 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-f581aab3-5a19-46e6-82d6-5fcef92b902c None None] MFA Rules not processed for user `ff4736f3d50746b0bf5d6a917b2ca0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:35:14.871888 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-f581aab3-5a19-46e6-82d6-5fcef92b902c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:35:14.957609 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1135/2249] 10.222.0.18 () {58 vars in 967 bytes} [Wed Nov 2 07:35:14 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:35:14.970512 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495f5c6c-fce9-4c85-868d-ea2844d0c9d2 service nova] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:35:14.971930 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-495f5c6c-fce9-4c85-868d-ea2844d0c9d2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:35:15.107069 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495f5c6c-fce9-4c85-868d-ea2844d0c9d2 service nova] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:35:15.112093 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a70819-3ded-4060-b20c-1fdc553bdc76 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9116f47889f4db39e0203d6cdae1593', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:35:15.112703 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a70819-3ded-4060-b20c-1fdc553bdc76 service swift] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:15.113012 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a70819-3ded-4060-b20c-1fdc553bdc76 service swift] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:15.113285 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a70819-3ded-4060-b20c-1fdc553bdc76 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:15.122479 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a70819-3ded-4060-b20c-1fdc553bdc76 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:35:15.124020 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a70819-3ded-4060-b20c-1fdc553bdc76 service swift] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:35:15.134231 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1115/2250] 10.222.0.18 () {60 vars in 1384 bytes} [Wed Nov 2 07:35:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Nov 02 07:35:24.949854 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=25 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:35:25.129587 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=90 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:35:38.306206 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f581aab3-5a19-46e6-82d6-5fcef92b902c None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:35:38.368580 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f581aab3-5a19-46e6-82d6-5fcef92b902c None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:35:38.373707 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28de1275-4fcb-47be-bc41-91132754897c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:35:38.374326 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28de1275-4fcb-47be-bc41-91132754897c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:38.374633 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28de1275-4fcb-47be-bc41-91132754897c admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:38.374905 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28de1275-4fcb-47be-bc41-91132754897c admin admin] PATH_INFO: `/v3/users/76946b04f98b47749cd2d8da5a2dae45` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:38.376026 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28de1275-4fcb-47be-bc41-91132754897c admin admin] RBAC: Authorizing `identity:delete_user(user_id=76946b04f98b47749cd2d8da5a2dae45)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:35:38.377923 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28de1275-4fcb-47be-bc41-91132754897c admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:35:38.424450 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-28de1275-4fcb-47be-bc41-91132754897c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '76946b04f98b47749cd2d8da5a2dae45'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:35:38.432526 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-28de1275-4fcb-47be-bc41-91132754897c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '76946b04f98b47749cd2d8da5a2dae45'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:35:38.433177 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-28de1275-4fcb-47be-bc41-91132754897c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '76946b04f98b47749cd2d8da5a2dae45'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:35:38.440074 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-28de1275-4fcb-47be-bc41-91132754897c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '76946b04f98b47749cd2d8da5a2dae45'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:35:38.440693 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-28de1275-4fcb-47be-bc41-91132754897c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '76946b04f98b47749cd2d8da5a2dae45'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:35:38.458612 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1136/2251] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Nov 2 07:35:38 2022] DELETE /identity/v3/users/76946b04f98b47749cd2d8da5a2dae45 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:35:38.463286 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1a89cb-7ed2-4199-96ad-3865caea61c5 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:38.463773 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1a89cb-7ed2-4199-96ad-3865caea61c5 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:38.463987 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1a89cb-7ed2-4199-96ad-3865caea61c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:38.482734 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-fe1a89cb-7ed2-4199-96ad-3865caea61c5 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:35:38.483903 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-fe1a89cb-7ed2-4199-96ad-3865caea61c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:35:38.543396 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1116/2252] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:35:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:35:38.553257 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28de1275-4fcb-47be-bc41-91132754897c admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:35:38.554709 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-28de1275-4fcb-47be-bc41-91132754897c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:35:38.612799 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28de1275-4fcb-47be-bc41-91132754897c admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:35:38.616464 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf36e05-c701-41d5-80e3-f4d85d0be108 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3ec6ecb85f94ef59762877d9ea33ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:35:38.617087 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf36e05-c701-41d5-80e3-f4d85d0be108 service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:38.617364 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf36e05-c701-41d5-80e3-f4d85d0be108 service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:38.617632 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf36e05-c701-41d5-80e3-f4d85d0be108 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:38.627222 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf36e05-c701-41d5-80e3-f4d85d0be108 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:35:38.628626 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf36e05-c701-41d5-80e3-f4d85d0be108 service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:35:38.638841 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1137/2253] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:35:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:35:38.905855 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1a89cb-7ed2-4199-96ad-3865caea61c5 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:35:38.907734 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-fe1a89cb-7ed2-4199-96ad-3865caea61c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:35:38.920535 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1a89cb-7ed2-4199-96ad-3865caea61c5 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:35:38.925509 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3740593f-79fc-485c-8021-9a72ee0ab856 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:35:38.926150 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3740593f-79fc-485c-8021-9a72ee0ab856 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:38.926466 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3740593f-79fc-485c-8021-9a72ee0ab856 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:38.926744 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3740593f-79fc-485c-8021-9a72ee0ab856 admin admin] PATH_INFO: `/v3/projects/cfd4f4705f464cd2ab4f3c35747b169f` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:38.927861 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3740593f-79fc-485c-8021-9a72ee0ab856 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cfd4f4705f464cd2ab4f3c35747b169f)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:35:38.929930 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3740593f-79fc-485c-8021-9a72ee0ab856 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:35:38.977458 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-3740593f-79fc-485c-8021-9a72ee0ab856 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cfd4f4705f464cd2ab4f3c35747b169f', 'request_id': 'req-3740593f-79fc-485c-8021-9a72ee0ab856'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:35:38.983633 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-3740593f-79fc-485c-8021-9a72ee0ab856 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cfd4f4705f464cd2ab4f3c35747b169f', 'request_id': 'req-3740593f-79fc-485c-8021-9a72ee0ab856'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:35:38.991198 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-3740593f-79fc-485c-8021-9a72ee0ab856 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cfd4f4705f464cd2ab4f3c35747b169f', 'request_id': 'req-3740593f-79fc-485c-8021-9a72ee0ab856'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:35:39.001429 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-3740593f-79fc-485c-8021-9a72ee0ab856 admin admin] The token cache is being invalidate because project cfd4f4705f464cd2ab4f3c35747b169f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81501) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:35:39.001713 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-3740593f-79fc-485c-8021-9a72ee0ab856 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:35:39.002346 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-3740593f-79fc-485c-8021-9a72ee0ab856 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:35:39.003824 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1117/2254] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Nov 2 07:35:38 2022] DELETE /identity/v3/projects/cfd4f4705f464cd2ab4f3c35747b169f => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:35:39.008916 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2576740-32dc-4257-944b-c4fdc690acde None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:39.009204 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2576740-32dc-4257-944b-c4fdc690acde None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:39.009478 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2576740-32dc-4257-944b-c4fdc690acde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:39.029734 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-e2576740-32dc-4257-944b-c4fdc690acde None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:35:39.030958 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-e2576740-32dc-4257-944b-c4fdc690acde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:35:39.047548 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:35:39.091930 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1138/2255] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:35:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:35:39.099418 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584f1cf1-f9a7-4075-bab3-7461241d9d0b None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:39.099705 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584f1cf1-f9a7-4075-bab3-7461241d9d0b None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:39.099980 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584f1cf1-f9a7-4075-bab3-7461241d9d0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:39.120536 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-584f1cf1-f9a7-4075-bab3-7461241d9d0b None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:35:39.121722 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-584f1cf1-f9a7-4075-bab3-7461241d9d0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:35:39.129729 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1118/2256] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:35:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:35:39.133945 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2576740-32dc-4257-944b-c4fdc690acde None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:35:39.143504 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2576740-32dc-4257-944b-c4fdc690acde None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:35:39.148128 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124241de-117b-4ee4-a750-c4f0170a3296 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:35:39.148720 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124241de-117b-4ee4-a750-c4f0170a3296 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:39.148996 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124241de-117b-4ee4-a750-c4f0170a3296 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:39.149265 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124241de-117b-4ee4-a750-c4f0170a3296 admin admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:39.149905 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124241de-117b-4ee4-a750-c4f0170a3296 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:35:39.150202 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124241de-117b-4ee4-a750-c4f0170a3296 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:35:39.152467 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124241de-117b-4ee4-a750-c4f0170a3296 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:35:39.163710 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1139/2257] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Nov 2 07:35:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:35:39.168745 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584f1cf1-f9a7-4075-bab3-7461241d9d0b None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:35:39.179096 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584f1cf1-f9a7-4075-bab3-7461241d9d0b None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:35:39.184440 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203d44f0-c454-43be-9b17-8fa1def893e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:35:39.185054 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203d44f0-c454-43be-9b17-8fa1def893e9 admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:39.185319 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203d44f0-c454-43be-9b17-8fa1def893e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:39.185590 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203d44f0-c454-43be-9b17-8fa1def893e9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:39.186225 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203d44f0-c454-43be-9b17-8fa1def893e9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:35:39.188151 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203d44f0-c454-43be-9b17-8fa1def893e9 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:35:39.215473 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1119/2258] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Nov 2 07:35:39 2022] POST /identity/v3/projects => generated 391 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 07:35:39.219765 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124241de-117b-4ee4-a750-c4f0170a3296 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:35:39.230896 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124241de-117b-4ee4-a750-c4f0170a3296 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:35:39.286664 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c936151a-0077-4e6b-9bd0-559d97d8c2b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:35:39.287260 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c936151a-0077-4e6b-9bd0-559d97d8c2b5 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:39.287542 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c936151a-0077-4e6b-9bd0-559d97d8c2b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:39.287812 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c936151a-0077-4e6b-9bd0-559d97d8c2b5 admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:39.288433 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c936151a-0077-4e6b-9bd0-559d97d8c2b5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:35:39.290332 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c936151a-0077-4e6b-9bd0-559d97d8c2b5 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:35:39.312193 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1140/2259] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Nov 2 07:35:39 2022] POST /identity/v3/users => generated 296 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:35:39.316314 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203d44f0-c454-43be-9b17-8fa1def893e9 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:35:39.326842 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203d44f0-c454-43be-9b17-8fa1def893e9 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:35:39.331505 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c897fbc-f06e-452c-9063-34c1854daaf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:35:39.332090 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c897fbc-f06e-452c-9063-34c1854daaf5 admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:39.332366 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c897fbc-f06e-452c-9063-34c1854daaf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:39.332632 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c897fbc-f06e-452c-9063-34c1854daaf5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:39.333258 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c897fbc-f06e-452c-9063-34c1854daaf5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:35:39.334867 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c897fbc-f06e-452c-9063-34c1854daaf5 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:35:39.345144 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1120/2260] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:35:39 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 07:35:39.348987 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c936151a-0077-4e6b-9bd0-559d97d8c2b5 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:35:39.359551 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c936151a-0077-4e6b-9bd0-559d97d8c2b5 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:35:39.364066 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d91729-0d81-4225-9d14-710d024f1446 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:35:39.364652 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d91729-0d81-4225-9d14-710d024f1446 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:39.364928 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d91729-0d81-4225-9d14-710d024f1446 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:39.365196 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d91729-0d81-4225-9d14-710d024f1446 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:39.365819 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d91729-0d81-4225-9d14-710d024f1446 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:35:39.367417 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d91729-0d81-4225-9d14-710d024f1446 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:35:39.378047 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1141/2261] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:35:39 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:35:39.382299 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c897fbc-f06e-452c-9063-34c1854daaf5 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:35:39.392147 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c897fbc-f06e-452c-9063-34c1854daaf5 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:35:39.396667 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d789ad-9ad0-401d-9378-f16ba04039e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:35:39.397295 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d789ad-9ad0-401d-9378-f16ba04039e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:39.397573 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d789ad-9ad0-401d-9378-f16ba04039e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:39.397863 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d789ad-9ad0-401d-9378-f16ba04039e5 admin admin] PATH_INFO: `/v3/projects/4154fbb813af483caead760ada3f005e/users/1b1b9688a5734acea0ccb845ec4f64f6/roles/b494fedd00f24f0b84a529e8672a156a` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:39.412864 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d789ad-9ad0-401d-9378-f16ba04039e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4154fbb813af483caead760ada3f005e, user_id=1b1b9688a5734acea0ccb845ec4f64f6, role_id=b494fedd00f24f0b84a529e8672a156a)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:35:39.415524 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d789ad-9ad0-401d-9378-f16ba04039e5 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:35:39.428763 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1121/2262] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Nov 2 07:35:39 2022] PUT /identity/v3/projects/4154fbb813af483caead760ada3f005e/users/1b1b9688a5734acea0ccb845ec4f64f6/roles/b494fedd00f24f0b84a529e8672a156a => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 07:35:39.440507 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf68f92-0699-4738-ac53-588aac792628 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:39.440791 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf68f92-0699-4738-ac53-588aac792628 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:39.441068 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf68f92-0699-4738-ac53-588aac792628 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:39.469278 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-fdf68f92-0699-4738-ac53-588aac792628 None None] MFA Rules not processed for user `1b1b9688a5734acea0ccb845ec4f64f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:35:39.470506 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-fdf68f92-0699-4738-ac53-588aac792628 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:35:39.533598 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1142/2263] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:35:39 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:35:39.544553 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d789ad-9ad0-401d-9378-f16ba04039e5 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:35:39.546256 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-e4d789ad-9ad0-401d-9378-f16ba04039e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:35:39.568105 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:35:39.604859 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d789ad-9ad0-401d-9378-f16ba04039e5 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:35:39.608425 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c080c552-033c-47d0-8220-bf0474ddb062 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43bb326fb82e4d2c83fb2eaf91860dc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:35:39.609018 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c080c552-033c-47d0-8220-bf0474ddb062 service cinder] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:39.609295 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c080c552-033c-47d0-8220-bf0474ddb062 service cinder] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:39.609563 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c080c552-033c-47d0-8220-bf0474ddb062 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:39.617578 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c080c552-033c-47d0-8220-bf0474ddb062 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:35:39.619049 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c080c552-033c-47d0-8220-bf0474ddb062 service cinder] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:35:39.627675 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1122/2264] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:35:39 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:35:49.530781 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=89,keystone:INSERT=3 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:35:49.623789 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=17 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:35:56.864120 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf68f92-0699-4738-ac53-588aac792628 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:35:56.925218 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf68f92-0699-4738-ac53-588aac792628 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:35:56.929698 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be3dac9-02c1-4df4-85af-45ca1290dd40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:35:56.930344 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be3dac9-02c1-4df4-85af-45ca1290dd40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:56.930657 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be3dac9-02c1-4df4-85af-45ca1290dd40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:56.930931 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be3dac9-02c1-4df4-85af-45ca1290dd40 admin admin] PATH_INFO: `/v3/users/1b1b9688a5734acea0ccb845ec4f64f6` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:56.932031 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be3dac9-02c1-4df4-85af-45ca1290dd40 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b1b9688a5734acea0ccb845ec4f64f6)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:35:56.933926 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be3dac9-02c1-4df4-85af-45ca1290dd40 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:35:56.979354 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-8be3dac9-02c1-4df4-85af-45ca1290dd40 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b1b9688a5734acea0ccb845ec4f64f6'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:35:56.987262 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-8be3dac9-02c1-4df4-85af-45ca1290dd40 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b1b9688a5734acea0ccb845ec4f64f6'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:35:56.987830 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-8be3dac9-02c1-4df4-85af-45ca1290dd40 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b1b9688a5734acea0ccb845ec4f64f6'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:35:56.994865 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-8be3dac9-02c1-4df4-85af-45ca1290dd40 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b1b9688a5734acea0ccb845ec4f64f6'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:35:56.995486 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-8be3dac9-02c1-4df4-85af-45ca1290dd40 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b1b9688a5734acea0ccb845ec4f64f6'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:35:57.013076 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1143/2265] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Nov 2 07:35:56 2022] DELETE /identity/v3/users/1b1b9688a5734acea0ccb845ec4f64f6 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:35:57.017725 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391701ca-d3f8-4404-a8f8-b5e79d00aca0 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:57.018213 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391701ca-d3f8-4404-a8f8-b5e79d00aca0 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:57.018309 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391701ca-d3f8-4404-a8f8-b5e79d00aca0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:57.037245 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-391701ca-d3f8-4404-a8f8-b5e79d00aca0 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:35:57.038484 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-391701ca-d3f8-4404-a8f8-b5e79d00aca0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:35:57.095999 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1123/2266] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:35:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:35:57.105677 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be3dac9-02c1-4df4-85af-45ca1290dd40 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:35:57.107127 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-8be3dac9-02c1-4df4-85af-45ca1290dd40 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:35:57.167667 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be3dac9-02c1-4df4-85af-45ca1290dd40 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:35:57.171182 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ce2da0-fff0-438c-857d-576d6b828bd6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3ec6ecb85f94ef59762877d9ea33ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:35:57.171776 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ce2da0-fff0-438c-857d-576d6b828bd6 service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:57.172681 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ce2da0-fff0-438c-857d-576d6b828bd6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:57.172949 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ce2da0-fff0-438c-857d-576d6b828bd6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:57.181688 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ce2da0-fff0-438c-857d-576d6b828bd6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:35:57.183125 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ce2da0-fff0-438c-857d-576d6b828bd6 service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:35:57.192356 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1144/2267] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:35:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 07:35:57.461603 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391701ca-d3f8-4404-a8f8-b5e79d00aca0 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:35:57.463182 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-391701ca-d3f8-4404-a8f8-b5e79d00aca0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:35:57.478947 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391701ca-d3f8-4404-a8f8-b5e79d00aca0 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:35:57.483773 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81a3d0b-2c3c-41f4-b1f9-091818fd704f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:35:57.484560 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81a3d0b-2c3c-41f4-b1f9-091818fd704f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:57.484915 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81a3d0b-2c3c-41f4-b1f9-091818fd704f admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:57.485190 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81a3d0b-2c3c-41f4-b1f9-091818fd704f admin admin] PATH_INFO: `/v3/projects/4154fbb813af483caead760ada3f005e` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:57.487030 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81a3d0b-2c3c-41f4-b1f9-091818fd704f admin admin] RBAC: Authorizing `identity:delete_project(project_id=4154fbb813af483caead760ada3f005e)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:35:57.489019 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81a3d0b-2c3c-41f4-b1f9-091818fd704f admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:35:57.536422 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-e81a3d0b-2c3c-41f4-b1f9-091818fd704f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4154fbb813af483caead760ada3f005e', 'request_id': 'req-e81a3d0b-2c3c-41f4-b1f9-091818fd704f'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:35:57.542509 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-e81a3d0b-2c3c-41f4-b1f9-091818fd704f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4154fbb813af483caead760ada3f005e', 'request_id': 'req-e81a3d0b-2c3c-41f4-b1f9-091818fd704f'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:35:57.549617 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-e81a3d0b-2c3c-41f4-b1f9-091818fd704f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4154fbb813af483caead760ada3f005e', 'request_id': 'req-e81a3d0b-2c3c-41f4-b1f9-091818fd704f'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:35:57.559771 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-e81a3d0b-2c3c-41f4-b1f9-091818fd704f admin admin] The token cache is being invalidate because project 4154fbb813af483caead760ada3f005e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81501) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:35:57.560054 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-e81a3d0b-2c3c-41f4-b1f9-091818fd704f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:35:57.560671 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-e81a3d0b-2c3c-41f4-b1f9-091818fd704f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:35:57.562131 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1124/2268] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Nov 2 07:35:57 2022] DELETE /identity/v3/projects/4154fbb813af483caead760ada3f005e => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:35:57.567151 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3a184d-6ca6-42e3-b8af-adfb21874223 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:57.567434 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3a184d-6ca6-42e3-b8af-adfb21874223 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:57.567706 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3a184d-6ca6-42e3-b8af-adfb21874223 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:57.587188 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-ab3a184d-6ca6-42e3-b8af-adfb21874223 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:35:57.588377 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-ab3a184d-6ca6-42e3-b8af-adfb21874223 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:35:57.605215 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:35:57.653182 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1145/2269] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:35:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:35:57.662000 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0023ee-af16-412b-bf52-bb33f0a36277 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:57.662286 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0023ee-af16-412b-bf52-bb33f0a36277 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:57.662606 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0023ee-af16-412b-bf52-bb33f0a36277 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:57.682277 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-4d0023ee-af16-412b-bf52-bb33f0a36277 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:35:57.683478 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-4d0023ee-af16-412b-bf52-bb33f0a36277 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:35:57.691296 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1125/2270] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:35:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:35:57.695346 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3a184d-6ca6-42e3-b8af-adfb21874223 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:35:57.705610 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3a184d-6ca6-42e3-b8af-adfb21874223 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:35:57.711142 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d21889e-a293-4756-90bf-01e8e17fbabe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:35:57.711745 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d21889e-a293-4756-90bf-01e8e17fbabe admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:57.712021 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d21889e-a293-4756-90bf-01e8e17fbabe admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:57.712289 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d21889e-a293-4756-90bf-01e8e17fbabe admin admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:57.712929 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d21889e-a293-4756-90bf-01e8e17fbabe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:35:57.713225 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d21889e-a293-4756-90bf-01e8e17fbabe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:35:57.714863 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d21889e-a293-4756-90bf-01e8e17fbabe admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:35:57.727883 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1146/2271] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Nov 2 07:35:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:35:57.732180 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0023ee-af16-412b-bf52-bb33f0a36277 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:35:57.741643 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0023ee-af16-412b-bf52-bb33f0a36277 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:35:57.746560 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef449db-9507-46e8-944b-b0e03ef12608 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:35:57.747145 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef449db-9507-46e8-944b-b0e03ef12608 admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:57.747425 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef449db-9507-46e8-944b-b0e03ef12608 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:57.747698 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef449db-9507-46e8-944b-b0e03ef12608 admin admin] PATH_INFO: `/v3/projects` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:57.748332 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef449db-9507-46e8-944b-b0e03ef12608 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:35:57.750994 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef449db-9507-46e8-944b-b0e03ef12608 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:35:57.777942 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1126/2272] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Nov 2 07:35:57 2022] POST /identity/v3/projects => generated 385 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:35:57.782383 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d21889e-a293-4756-90bf-01e8e17fbabe admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:35:57.792072 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d21889e-a293-4756-90bf-01e8e17fbabe admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:35:57.848062 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8433340f-fd75-4c81-acbc-1396bf1fd54d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:35:57.848743 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8433340f-fd75-4c81-acbc-1396bf1fd54d admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:57.849023 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8433340f-fd75-4c81-acbc-1396bf1fd54d admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:57.849322 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8433340f-fd75-4c81-acbc-1396bf1fd54d admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:57.849963 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8433340f-fd75-4c81-acbc-1396bf1fd54d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:35:57.851918 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8433340f-fd75-4c81-acbc-1396bf1fd54d admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:35:57.872078 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1147/2273] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Nov 2 07:35:57 2022] POST /identity/v3/users => generated 293 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 07:35:57.876644 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef449db-9507-46e8-944b-b0e03ef12608 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:35:57.887012 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef449db-9507-46e8-944b-b0e03ef12608 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:35:57.891644 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e349d4c1-d75e-4ec6-9c84-a8ec148de642 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:35:57.892232 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e349d4c1-d75e-4ec6-9c84-a8ec148de642 admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:57.892512 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e349d4c1-d75e-4ec6-9c84-a8ec148de642 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:57.892781 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e349d4c1-d75e-4ec6-9c84-a8ec148de642 admin admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:57.893414 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e349d4c1-d75e-4ec6-9c84-a8ec148de642 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:35:57.895079 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e349d4c1-d75e-4ec6-9c84-a8ec148de642 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:35:57.904704 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1127/2274] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:35:57 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:35:57.909087 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8433340f-fd75-4c81-acbc-1396bf1fd54d admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:35:57.918779 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8433340f-fd75-4c81-acbc-1396bf1fd54d admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:35:57.923869 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c592c6-71ae-4253-a99e-4f82bb47a478 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:35:57.924456 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c592c6-71ae-4253-a99e-4f82bb47a478 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:57.924737 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c592c6-71ae-4253-a99e-4f82bb47a478 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:57.925011 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c592c6-71ae-4253-a99e-4f82bb47a478 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:57.925641 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c592c6-71ae-4253-a99e-4f82bb47a478 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:35:57.927250 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c592c6-71ae-4253-a99e-4f82bb47a478 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:35:57.937659 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1148/2275] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:35:57 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:35:57.941668 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e349d4c1-d75e-4ec6-9c84-a8ec148de642 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:35:57.951713 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e349d4c1-d75e-4ec6-9c84-a8ec148de642 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:35:57.956318 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c209e48-ab99-4115-a120-034b5f33414b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:35:57.956900 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c209e48-ab99-4115-a120-034b5f33414b admin admin] REQUEST_METHOD: `PUT` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:57.957179 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c209e48-ab99-4115-a120-034b5f33414b admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:57.957451 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c209e48-ab99-4115-a120-034b5f33414b admin admin] PATH_INFO: `/v3/projects/280ce4c6b9174073902820e2e922bedf/users/b0d0493e92b746eabdd4cb0912fa250c/roles/b494fedd00f24f0b84a529e8672a156a` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:57.972342 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c209e48-ab99-4115-a120-034b5f33414b admin admin] RBAC: Authorizing `identity:create_grant(project_id=280ce4c6b9174073902820e2e922bedf, user_id=b0d0493e92b746eabdd4cb0912fa250c, role_id=b494fedd00f24f0b84a529e8672a156a)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:35:57.975031 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c209e48-ab99-4115-a120-034b5f33414b admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:35:57.988357 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1128/2276] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Nov 2 07:35:57 2022] PUT /identity/v3/projects/280ce4c6b9174073902820e2e922bedf/users/b0d0493e92b746eabdd4cb0912fa250c/roles/b494fedd00f24f0b84a529e8672a156a => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:35:57.993439 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8226d104-df77-4851-9842-a02f9ac6a867 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:57.993778 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8226d104-df77-4851-9842-a02f9ac6a867 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:57.994130 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8226d104-df77-4851-9842-a02f9ac6a867 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:58.014568 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-8226d104-df77-4851-9842-a02f9ac6a867 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:35:58.015832 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-8226d104-df77-4851-9842-a02f9ac6a867 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:35:58.070138 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c209e48-ab99-4115-a120-034b5f33414b admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:35:58.071896 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-8c209e48-ab99-4115-a120-034b5f33414b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:35:58.075343 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1149/2277] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:35:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:35:58.084958 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8226d104-df77-4851-9842-a02f9ac6a867 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:35:58.086264 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-8226d104-df77-4851-9842-a02f9ac6a867 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:35:58.094918 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:35:58.118626 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:35:58.134259 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c209e48-ab99-4115-a120-034b5f33414b admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:35:58.138584 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7a1f5b-eb0f-4ac2-80d0-8c7e18ad2973 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43bb326fb82e4d2c83fb2eaf91860dc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:35:58.139286 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7a1f5b-eb0f-4ac2-80d0-8c7e18ad2973 service cinder] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:58.139719 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7a1f5b-eb0f-4ac2-80d0-8c7e18ad2973 service cinder] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:58.140083 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7a1f5b-eb0f-4ac2-80d0-8c7e18ad2973 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:58.143191 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-2d7a1f5b-eb0f-4ac2-80d0-8c7e18ad2973 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:35:58.146678 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8226d104-df77-4851-9842-a02f9ac6a867 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:35:58.150238 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35c8a43-370e-43ad-a205-de46f289d07d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3ec6ecb85f94ef59762877d9ea33ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:35:58.150867 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35c8a43-370e-43ad-a205-de46f289d07d service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:58.151139 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35c8a43-370e-43ad-a205-de46f289d07d service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:58.151402 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35c8a43-370e-43ad-a205-de46f289d07d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:58.161103 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35c8a43-370e-43ad-a205-de46f289d07d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:35:58.163449 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35c8a43-370e-43ad-a205-de46f289d07d service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:35:58.173492 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1150/2278] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:35:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 07:35:58.202835 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7a1f5b-eb0f-4ac2-80d0-8c7e18ad2973 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:35:58.205212 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7a1f5b-eb0f-4ac2-80d0-8c7e18ad2973 service cinder] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:35:58.214351 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1129/2279] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:35:58 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 07:36:02.516109 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb403ba-9fcb-4634-afe8-eebe54f7e08a None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:02.517177 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb403ba-9fcb-4634-afe8-eebe54f7e08a None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:02.517869 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb403ba-9fcb-4634-afe8-eebe54f7e08a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:02.556795 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-7eb403ba-9fcb-4634-afe8-eebe54f7e08a None None] MFA Rules not processed for user `b0d0493e92b746eabdd4cb0912fa250c`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:36:02.558003 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-7eb403ba-9fcb-4634-afe8-eebe54f7e08a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:36:02.629390 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1151/2280] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:36:02 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:36:02.634176 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7a1f5b-eb0f-4ac2-80d0-8c7e18ad2973 service cinder] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:36:02.647100 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7a1f5b-eb0f-4ac2-80d0-8c7e18ad2973 service cinder] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:36:02.651736 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f64bc94-67c9-48b2-b50a-209a0cd8dc1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:36:02.652352 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f64bc94-67c9-48b2-b50a-209a0cd8dc1a admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:02.652634 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f64bc94-67c9-48b2-b50a-209a0cd8dc1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:02.652905 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f64bc94-67c9-48b2-b50a-209a0cd8dc1a admin admin] PATH_INFO: `/v3/projects` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:02.653537 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f64bc94-67c9-48b2-b50a-209a0cd8dc1a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:36:02.655664 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f64bc94-67c9-48b2-b50a-209a0cd8dc1a admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:36:02.692394 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1130/2281] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Nov 2 07:36:02 2022] POST /identity/v3/projects => generated 385 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:36:02.693622 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb403ba-9fcb-4634-afe8-eebe54f7e08a None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:36:02.706876 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb403ba-9fcb-4634-afe8-eebe54f7e08a None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:36:02.770823 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387ea218-0c05-4076-8280-18efa493e6c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:36:02.771528 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387ea218-0c05-4076-8280-18efa493e6c3 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:02.771881 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387ea218-0c05-4076-8280-18efa493e6c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:02.772220 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387ea218-0c05-4076-8280-18efa493e6c3 admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:02.772922 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387ea218-0c05-4076-8280-18efa493e6c3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:36:02.774875 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387ea218-0c05-4076-8280-18efa493e6c3 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:36:02.802692 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1152/2282] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Nov 2 07:36:02 2022] POST /identity/v3/users => generated 299 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 07:36:02.803570 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f64bc94-67c9-48b2-b50a-209a0cd8dc1a admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:36:02.814498 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f64bc94-67c9-48b2-b50a-209a0cd8dc1a admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:36:02.822208 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8018efaf-2a42-4515-95ee-f154ec78e809 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:36:02.822970 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8018efaf-2a42-4515-95ee-f154ec78e809 admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:02.823440 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8018efaf-2a42-4515-95ee-f154ec78e809 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:02.823812 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8018efaf-2a42-4515-95ee-f154ec78e809 admin admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:02.824654 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8018efaf-2a42-4515-95ee-f154ec78e809 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:36:02.826604 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8018efaf-2a42-4515-95ee-f154ec78e809 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:36:02.839314 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1131/2283] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:36:02 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 07:36:02.843588 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387ea218-0c05-4076-8280-18efa493e6c3 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:36:02.856347 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387ea218-0c05-4076-8280-18efa493e6c3 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:36:02.861972 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a76e2a-4bf4-4064-8274-3cc8fed4f9ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:36:02.863126 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a76e2a-4bf4-4064-8274-3cc8fed4f9ef admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:02.863241 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a76e2a-4bf4-4064-8274-3cc8fed4f9ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:02.863593 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a76e2a-4bf4-4064-8274-3cc8fed4f9ef admin admin] PATH_INFO: `/v3/projects/546abb669f364409a3d449dd8056a2c6/users/4735f72a2f1c4dbdb6b244aadeb71d00/roles/c4bf019fc6c44c71a6c984f6d0bab75f` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:02.882466 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a76e2a-4bf4-4064-8274-3cc8fed4f9ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=546abb669f364409a3d449dd8056a2c6, user_id=4735f72a2f1c4dbdb6b244aadeb71d00, role_id=c4bf019fc6c44c71a6c984f6d0bab75f)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:36:02.885125 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a76e2a-4bf4-4064-8274-3cc8fed4f9ef admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:36:02.912637 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1153/2284] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Nov 2 07:36:02 2022] PUT /identity/v3/projects/546abb669f364409a3d449dd8056a2c6/users/4735f72a2f1c4dbdb6b244aadeb71d00/roles/c4bf019fc6c44c71a6c984f6d0bab75f => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:36:02.917065 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8018efaf-2a42-4515-95ee-f154ec78e809 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:36:02.932269 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8018efaf-2a42-4515-95ee-f154ec78e809 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:36:03.028655 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bd98e7-888b-411d-af11-b347c2966af5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:36:03.029525 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bd98e7-888b-411d-af11-b347c2966af5 admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:03.029908 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bd98e7-888b-411d-af11-b347c2966af5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:03.030280 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bd98e7-888b-411d-af11-b347c2966af5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:03.031059 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bd98e7-888b-411d-af11-b347c2966af5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:36:03.033792 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bd98e7-888b-411d-af11-b347c2966af5 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:36:03.045160 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1132/2285] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:36:02 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:36:03.051334 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a76e2a-4bf4-4064-8274-3cc8fed4f9ef admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:36:03.062081 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a76e2a-4bf4-4064-8274-3cc8fed4f9ef admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:36:03.068236 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a53e257-ffcc-4298-a6f0-f067d79f06bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:36:03.069021 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a53e257-ffcc-4298-a6f0-f067d79f06bb admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:03.069429 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a53e257-ffcc-4298-a6f0-f067d79f06bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:03.069884 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a53e257-ffcc-4298-a6f0-f067d79f06bb admin admin] PATH_INFO: `/v3/projects/546abb669f364409a3d449dd8056a2c6/users/4735f72a2f1c4dbdb6b244aadeb71d00/roles/c4bf019fc6c44c71a6c984f6d0bab75f` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:03.072068 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a53e257-ffcc-4298-a6f0-f067d79f06bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=546abb669f364409a3d449dd8056a2c6, user_id=4735f72a2f1c4dbdb6b244aadeb71d00, role_id=c4bf019fc6c44c71a6c984f6d0bab75f)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:36:03.075142 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a53e257-ffcc-4298-a6f0-f067d79f06bb admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:36:03.087723 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1154/2286] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Nov 2 07:36:03 2022] PUT /identity/v3/projects/546abb669f364409a3d449dd8056a2c6/users/4735f72a2f1c4dbdb6b244aadeb71d00/roles/c4bf019fc6c44c71a6c984f6d0bab75f => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:36:05.343031 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07113d1-ac2b-450b-bb97-68ee8e4b1327 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:05.345088 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07113d1-ac2b-450b-bb97-68ee8e4b1327 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:05.345711 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07113d1-ac2b-450b-bb97-68ee8e4b1327 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:05.371477 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-b07113d1-ac2b-450b-bb97-68ee8e4b1327 None None] MFA Rules not processed for user `fe26aea6bb9d4dcebf046da719b2c244`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:36:05.376097 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-b07113d1-ac2b-450b-bb97-68ee8e4b1327 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:36:05.425210 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:36:05.494017 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1133/2287] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:36:05 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:36:06.304385 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a53e257-ffcc-4298-a6f0-f067d79f06bb admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:36:06.327934 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:36:06.408542 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a53e257-ffcc-4298-a6f0-f067d79f06bb admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:36:06.413496 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ff8cee-34ac-4747-9df7-cc18af97786e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3ec6ecb85f94ef59762877d9ea33ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:36:06.414221 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ff8cee-34ac-4747-9df7-cc18af97786e service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:06.415606 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ff8cee-34ac-4747-9df7-cc18af97786e service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:06.415955 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ff8cee-34ac-4747-9df7-cc18af97786e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:06.420287 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-74ff8cee-34ac-4747-9df7-cc18af97786e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:36:06.452182 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ff8cee-34ac-4747-9df7-cc18af97786e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:36:06.455166 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ff8cee-34ac-4747-9df7-cc18af97786e service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:36:06.467081 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1155/2288] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:36:06 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:36:09.133443 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15dc521-fd33-42c2-803f-98f59a6450d8 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:09.134029 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15dc521-fd33-42c2-803f-98f59a6450d8 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:09.134029 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15dc521-fd33-42c2-803f-98f59a6450d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:09.231207 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-b15dc521-fd33-42c2-803f-98f59a6450d8 None None] MFA Rules not processed for user `4735f72a2f1c4dbdb6b244aadeb71d00`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:36:09.240512 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-b15dc521-fd33-42c2-803f-98f59a6450d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:36:09.370649 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ff8cee-34ac-4747-9df7-cc18af97786e service neutron] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:36:09.371944 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-74ff8cee-34ac-4747-9df7-cc18af97786e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:36:09.441932 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1134/2289] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:36:09 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 308 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:36:09.448229 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15dc521-fd33-42c2-803f-98f59a6450d8 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:36:09.449655 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-b15dc521-fd33-42c2-803f-98f59a6450d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:36:09.539457 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ff8cee-34ac-4747-9df7-cc18af97786e service neutron] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:36:09.567432 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34d885e-b449-476f-bce7-43eb02adbc08 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e48cba3f804cf08b2477f01ae95179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:36:09.568059 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34d885e-b449-476f-bce7-43eb02adbc08 service nova] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:09.568338 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34d885e-b449-476f-bce7-43eb02adbc08 service nova] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:09.568846 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34d885e-b449-476f-bce7-43eb02adbc08 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:09.571649 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-a34d885e-b449-476f-bce7-43eb02adbc08 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:36:09.592142 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15dc521-fd33-42c2-803f-98f59a6450d8 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:36:09.604138 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ac986d-86f8-4db5-a067-283d53c55502 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5353f2af0f44d668e2a269647aa1dc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:36:09.604928 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ac986d-86f8-4db5-a067-283d53c55502 service glance] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:09.605278 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ac986d-86f8-4db5-a067-283d53c55502 service glance] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:09.605630 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ac986d-86f8-4db5-a067-283d53c55502 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:09.710658 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34d885e-b449-476f-bce7-43eb02adbc08 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:36:09.714563 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34d885e-b449-476f-bce7-43eb02adbc08 service nova] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:36:09.728196 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ac986d-86f8-4db5-a067-283d53c55502 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:36:09.731019 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ac986d-86f8-4db5-a067-283d53c55502 service glance] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:36:09.750069 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1156/2290] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:36:09 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 07:36:09.759302 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1135/2291] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:36:09 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:36:10.077230 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6139e9-8538-4baa-935b-f7a3f751765d None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:10.077984 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6139e9-8538-4baa-935b-f7a3f751765d None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:10.078432 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6139e9-8538-4baa-935b-f7a3f751765d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:10.117291 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-be6139e9-8538-4baa-935b-f7a3f751765d None None] MFA Rules not processed for user `ff4736f3d50746b0bf5d6a917b2ca0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:36:10.128438 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-be6139e9-8538-4baa-935b-f7a3f751765d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:36:10.258019 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:36:10.272224 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1157/2292] 10.222.0.18 () {58 vars in 1009 bytes} [Wed Nov 2 07:36:10 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:36:10.309226 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ac986d-86f8-4db5-a067-283d53c55502 service glance] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:36:10.311390 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-b3ac986d-86f8-4db5-a067-283d53c55502 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:36:10.418519 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=100 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:36:10.568571 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ac986d-86f8-4db5-a067-283d53c55502 service glance] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:36:10.581944 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4475b5-345b-4612-82a4-cd928750ab61 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9116f47889f4db39e0203d6cdae1593', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:36:10.583531 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4475b5-345b-4612-82a4-cd928750ab61 service swift] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:10.584047 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4475b5-345b-4612-82a4-cd928750ab61 service swift] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:10.584481 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4475b5-345b-4612-82a4-cd928750ab61 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:10.603878 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4475b5-345b-4612-82a4-cd928750ab61 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:36:10.606562 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4475b5-345b-4612-82a4-cd928750ab61 service swift] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:36:10.619716 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1136/2293] 10.222.0.18 () {60 vars in 1384 bytes} [Wed Nov 2 07:36:10 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Nov 02 07:36:14.276154 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6139e9-8538-4baa-935b-f7a3f751765d None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:36:14.289436 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6139e9-8538-4baa-935b-f7a3f751765d None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:36:14.293736 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b765d2fc-6a64-431f-9a74-7071b15ffbee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3ec6ecb85f94ef59762877d9ea33ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:36:14.294996 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b765d2fc-6a64-431f-9a74-7071b15ffbee service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:14.295271 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b765d2fc-6a64-431f-9a74-7071b15ffbee service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:14.295538 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b765d2fc-6a64-431f-9a74-7071b15ffbee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:14.297966 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-b765d2fc-6a64-431f-9a74-7071b15ffbee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:36:14.384498 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b765d2fc-6a64-431f-9a74-7071b15ffbee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:36:14.385948 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b765d2fc-6a64-431f-9a74-7071b15ffbee service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:36:14.397093 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1158/2294] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:36:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 07:36:18.611362 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4475b5-345b-4612-82a4-cd928750ab61 service swift] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:36:18.617948 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-4e4475b5-345b-4612-82a4-cd928750ab61 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:36:18.693725 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b765d2fc-6a64-431f-9a74-7071b15ffbee service neutron] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:36:18.695376 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-b765d2fc-6a64-431f-9a74-7071b15ffbee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:36:18.716684 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b765d2fc-6a64-431f-9a74-7071b15ffbee service neutron] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:36:18.725857 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be624d91-00ff-476b-9247-a79ed49e2302 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:36:18.726857 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be624d91-00ff-476b-9247-a79ed49e2302 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:18.727377 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be624d91-00ff-476b-9247-a79ed49e2302 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:18.727951 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be624d91-00ff-476b-9247-a79ed49e2302 admin admin] PATH_INFO: `/v3/users/2455ebfc92ba4d398a6279d6102a0408` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:18.729457 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be624d91-00ff-476b-9247-a79ed49e2302 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2455ebfc92ba4d398a6279d6102a0408)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:36:18.731675 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be624d91-00ff-476b-9247-a79ed49e2302 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:36:18.827268 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4475b5-345b-4612-82a4-cd928750ab61 service swift] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:36:18.833218 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0ff916-1e77-4170-b265-10eb0c4c0469 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2fc6a156c54d0c9d325901d0f7f6fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:36:18.833847 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0ff916-1e77-4170-b265-10eb0c4c0469 service placement] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:18.834119 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0ff916-1e77-4170-b265-10eb0c4c0469 service placement] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:18.834425 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0ff916-1e77-4170-b265-10eb0c4c0469 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:18.836343 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-ea0ff916-1e77-4170-b265-10eb0c4c0469 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:36:18.837810 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-be624d91-00ff-476b-9247-a79ed49e2302 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2455ebfc92ba4d398a6279d6102a0408'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:36:18.849879 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-be624d91-00ff-476b-9247-a79ed49e2302 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2455ebfc92ba4d398a6279d6102a0408'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:36:18.850123 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0ff916-1e77-4170-b265-10eb0c4c0469 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:36:18.850958 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-be624d91-00ff-476b-9247-a79ed49e2302 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2455ebfc92ba4d398a6279d6102a0408'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:36:18.853709 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0ff916-1e77-4170-b265-10eb0c4c0469 service placement] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:36:18.861266 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-be624d91-00ff-476b-9247-a79ed49e2302 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2455ebfc92ba4d398a6279d6102a0408'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:36:18.862309 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-be624d91-00ff-476b-9247-a79ed49e2302 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2455ebfc92ba4d398a6279d6102a0408'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:36:18.898102 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1137/2295] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:36:18 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 07:36:18.914634 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1159/2296] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Nov 2 07:36:18 2022] DELETE /identity/v3/users/2455ebfc92ba4d398a6279d6102a0408 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 07:36:19.143123 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0ff916-1e77-4170-b265-10eb0c4c0469 service placement] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:36:19.165400 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0ff916-1e77-4170-b265-10eb0c4c0469 service placement] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:36:19.255422 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e66516-8cfc-4dc2-8ca0-2186bc21bf95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:36:19.256419 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e66516-8cfc-4dc2-8ca0-2186bc21bf95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:19.256762 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e66516-8cfc-4dc2-8ca0-2186bc21bf95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:19.257138 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e66516-8cfc-4dc2-8ca0-2186bc21bf95 admin admin] PATH_INFO: `/v3/projects/df168e1504c947169b74f50c1ed4de95` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:19.258316 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e66516-8cfc-4dc2-8ca0-2186bc21bf95 admin admin] RBAC: Authorizing `identity:delete_project(project_id=df168e1504c947169b74f50c1ed4de95)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:36:19.260313 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e66516-8cfc-4dc2-8ca0-2186bc21bf95 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:36:19.314132 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:36:19.320885 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-53e66516-8cfc-4dc2-8ca0-2186bc21bf95 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'df168e1504c947169b74f50c1ed4de95', 'request_id': 'req-53e66516-8cfc-4dc2-8ca0-2186bc21bf95'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:36:19.328300 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-53e66516-8cfc-4dc2-8ca0-2186bc21bf95 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'df168e1504c947169b74f50c1ed4de95', 'request_id': 'req-53e66516-8cfc-4dc2-8ca0-2186bc21bf95'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:36:19.338292 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-53e66516-8cfc-4dc2-8ca0-2186bc21bf95 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'df168e1504c947169b74f50c1ed4de95', 'request_id': 'req-53e66516-8cfc-4dc2-8ca0-2186bc21bf95'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:36:19.351347 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-53e66516-8cfc-4dc2-8ca0-2186bc21bf95 admin admin] The token cache is being invalidate because project df168e1504c947169b74f50c1ed4de95 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81501) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:36:19.351676 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-53e66516-8cfc-4dc2-8ca0-2186bc21bf95 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:36:19.352259 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-53e66516-8cfc-4dc2-8ca0-2186bc21bf95 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:36:19.353806 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1138/2297] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Nov 2 07:36:19 2022] DELETE /identity/v3/projects/df168e1504c947169b74f50c1ed4de95 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:36:19.359013 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4ae2da-7d0d-404a-bbfc-7d83a9262058 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:19.359301 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4ae2da-7d0d-404a-bbfc-7d83a9262058 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:19.359576 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4ae2da-7d0d-404a-bbfc-7d83a9262058 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:19.381361 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-7e4ae2da-7d0d-404a-bbfc-7d83a9262058 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:36:19.382621 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-7e4ae2da-7d0d-404a-bbfc-7d83a9262058 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:36:19.458181 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1160/2298] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:36:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:36:19.466580 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075b8acc-4dc4-45a3-b7d8-90f269fd8769 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:19.466867 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075b8acc-4dc4-45a3-b7d8-90f269fd8769 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:19.467136 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075b8acc-4dc4-45a3-b7d8-90f269fd8769 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:19.494011 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-075b8acc-4dc4-45a3-b7d8-90f269fd8769 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:36:19.495702 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-075b8acc-4dc4-45a3-b7d8-90f269fd8769 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:36:19.505466 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1139/2299] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:36:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:36:19.511942 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4ae2da-7d0d-404a-bbfc-7d83a9262058 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:36:19.523340 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4ae2da-7d0d-404a-bbfc-7d83a9262058 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:36:19.524599 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:36:19.530854 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d2ab48-4062-4951-85c0-13e318fd8634 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:36:19.534874 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d2ab48-4062-4951-85c0-13e318fd8634 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:19.535291 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d2ab48-4062-4951-85c0-13e318fd8634 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:19.536062 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d2ab48-4062-4951-85c0-13e318fd8634 admin admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:19.536867 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d2ab48-4062-4951-85c0-13e318fd8634 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:36:19.537241 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d2ab48-4062-4951-85c0-13e318fd8634 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:36:19.538988 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d2ab48-4062-4951-85c0-13e318fd8634 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:36:19.559167 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1161/2300] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Nov 2 07:36:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:36:19.560599 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075b8acc-4dc4-45a3-b7d8-90f269fd8769 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:36:19.577968 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075b8acc-4dc4-45a3-b7d8-90f269fd8769 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:36:19.584663 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1ab243-c540-4eee-8591-8018be33e706 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:36:19.585360 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1ab243-c540-4eee-8591-8018be33e706 admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:19.585709 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1ab243-c540-4eee-8591-8018be33e706 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:19.585981 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1ab243-c540-4eee-8591-8018be33e706 admin admin] PATH_INFO: `/v3/projects` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:19.588825 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1ab243-c540-4eee-8591-8018be33e706 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:36:19.590782 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1ab243-c540-4eee-8591-8018be33e706 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:36:19.629965 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1140/2301] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Nov 2 07:36:19 2022] POST /identity/v3/projects => generated 375 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:36:19.634234 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d2ab48-4062-4951-85c0-13e318fd8634 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:36:19.663647 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d2ab48-4062-4951-85c0-13e318fd8634 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:36:19.746481 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166f04cc-2508-41cb-85a0-792b6f5e4e1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:36:19.746481 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166f04cc-2508-41cb-85a0-792b6f5e4e1b admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:19.746481 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166f04cc-2508-41cb-85a0-792b6f5e4e1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:19.746481 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166f04cc-2508-41cb-85a0-792b6f5e4e1b admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:19.747390 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166f04cc-2508-41cb-85a0-792b6f5e4e1b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:36:19.748773 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166f04cc-2508-41cb-85a0-792b6f5e4e1b admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:36:19.776191 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1162/2302] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Nov 2 07:36:19 2022] POST /identity/v3/users => generated 288 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:36:19.779889 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1ab243-c540-4eee-8591-8018be33e706 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:36:19.792923 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1ab243-c540-4eee-8591-8018be33e706 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:36:19.798642 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd075a09-1c6c-446f-a74f-01b8aa4c6262 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:36:19.799362 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd075a09-1c6c-446f-a74f-01b8aa4c6262 admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:19.799744 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd075a09-1c6c-446f-a74f-01b8aa4c6262 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:19.800248 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd075a09-1c6c-446f-a74f-01b8aa4c6262 admin admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:19.801036 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd075a09-1c6c-446f-a74f-01b8aa4c6262 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:36:19.802758 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd075a09-1c6c-446f-a74f-01b8aa4c6262 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:36:19.825861 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1141/2303] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:36:19 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:36:19.826176 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166f04cc-2508-41cb-85a0-792b6f5e4e1b admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:36:19.840406 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166f04cc-2508-41cb-85a0-792b6f5e4e1b admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:36:19.847362 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c267608-3ebf-4c21-9a11-279fce6d630b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:36:19.848074 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c267608-3ebf-4c21-9a11-279fce6d630b admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:19.848528 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c267608-3ebf-4c21-9a11-279fce6d630b admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:19.848901 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c267608-3ebf-4c21-9a11-279fce6d630b admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:19.849642 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c267608-3ebf-4c21-9a11-279fce6d630b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:36:19.853669 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c267608-3ebf-4c21-9a11-279fce6d630b admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:36:19.866530 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1163/2304] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:36:19 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:36:19.870667 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd075a09-1c6c-446f-a74f-01b8aa4c6262 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:36:19.884463 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd075a09-1c6c-446f-a74f-01b8aa4c6262 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:36:19.890391 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4919b612-9f94-4305-9165-2f408d8aa44f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:36:19.892036 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4919b612-9f94-4305-9165-2f408d8aa44f admin admin] REQUEST_METHOD: `PUT` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:19.892572 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4919b612-9f94-4305-9165-2f408d8aa44f admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:19.893002 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4919b612-9f94-4305-9165-2f408d8aa44f admin admin] PATH_INFO: `/v3/projects/86cca0dcb21a47e483458b36e33f5036/users/33c63dc278ac4eb79d2a4553fcbeda4f/roles/b494fedd00f24f0b84a529e8672a156a` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:19.915094 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4919b612-9f94-4305-9165-2f408d8aa44f admin admin] RBAC: Authorizing `identity:create_grant(project_id=86cca0dcb21a47e483458b36e33f5036, user_id=33c63dc278ac4eb79d2a4553fcbeda4f, role_id=b494fedd00f24f0b84a529e8672a156a)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:36:19.918475 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4919b612-9f94-4305-9165-2f408d8aa44f admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:36:19.935405 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1142/2305] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Nov 2 07:36:19 2022] PUT /identity/v3/projects/86cca0dcb21a47e483458b36e33f5036/users/33c63dc278ac4eb79d2a4553fcbeda4f/roles/b494fedd00f24f0b84a529e8672a156a => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:36:19.941076 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dbf158-f323-4d9a-ae4b-77f56e0c5a78 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:19.941367 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dbf158-f323-4d9a-ae4b-77f56e0c5a78 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:19.941643 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dbf158-f323-4d9a-ae4b-77f56e0c5a78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:19.980634 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-98dbf158-f323-4d9a-ae4b-77f56e0c5a78 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:36:19.982013 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-98dbf158-f323-4d9a-ae4b-77f56e0c5a78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:36:20.079452 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1164/2306] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:36:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:36:20.089884 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4919b612-9f94-4305-9165-2f408d8aa44f admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:36:20.092638 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-4919b612-9f94-4305-9165-2f408d8aa44f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:36:20.160005 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4919b612-9f94-4305-9165-2f408d8aa44f admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:36:20.163814 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a7bed5-73db-483c-8a21-d6799aa46d72 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3ec6ecb85f94ef59762877d9ea33ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:36:20.164496 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a7bed5-73db-483c-8a21-d6799aa46d72 service neutron] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:20.164842 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a7bed5-73db-483c-8a21-d6799aa46d72 service neutron] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:20.165382 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a7bed5-73db-483c-8a21-d6799aa46d72 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:20.179360 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a7bed5-73db-483c-8a21-d6799aa46d72 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:36:20.181139 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a7bed5-73db-483c-8a21-d6799aa46d72 service neutron] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:36:20.191992 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1143/2307] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:36:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:36:23.915659 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d35e386-69ad-4fb3-9adb-04a261850b1c None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:23.916621 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d35e386-69ad-4fb3-9adb-04a261850b1c None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:23.917103 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d35e386-69ad-4fb3-9adb-04a261850b1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:23.957139 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-2d35e386-69ad-4fb3-9adb-04a261850b1c None None] MFA Rules not processed for user `b0d0493e92b746eabdd4cb0912fa250c`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:36:23.959994 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-2d35e386-69ad-4fb3-9adb-04a261850b1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:36:24.099807 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1165/2308] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:36:23 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:36:26.411239 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e9e9a7-f1ac-4293-b43f-5c205f094d51 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:26.413263 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e9e9a7-f1ac-4293-b43f-5c205f094d51 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:26.413768 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e9e9a7-f1ac-4293-b43f-5c205f094d51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:26.466676 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-b5e9e9a7-f1ac-4293-b43f-5c205f094d51 None None] MFA Rules not processed for user `33c63dc278ac4eb79d2a4553fcbeda4f`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:36:26.467891 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-b5e9e9a7-f1ac-4293-b43f-5c205f094d51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:36:26.557650 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1144/2309] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:36:26 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:36:26.567302 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:36:26.576963 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d35e386-69ad-4fb3-9adb-04a261850b1c None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:36:26.579645 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-2d35e386-69ad-4fb3-9adb-04a261850b1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:36:26.611591 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:36:26.760321 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d35e386-69ad-4fb3-9adb-04a261850b1c None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:36:26.767887 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab525dcf-721a-43a1-8c21-327afec352a0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e48cba3f804cf08b2477f01ae95179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:36:26.768963 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab525dcf-721a-43a1-8c21-327afec352a0 service nova] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:26.769536 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab525dcf-721a-43a1-8c21-327afec352a0 service nova] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:26.770233 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab525dcf-721a-43a1-8c21-327afec352a0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:26.827189 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab525dcf-721a-43a1-8c21-327afec352a0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:36:26.830150 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab525dcf-721a-43a1-8c21-327afec352a0 service nova] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:36:26.841470 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1166/2310] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:36:26 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 266 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:36:27.837581 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87210fe-942f-4a96-99d7-bf068464669a None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:27.838633 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87210fe-942f-4a96-99d7-bf068464669a None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:27.839166 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87210fe-942f-4a96-99d7-bf068464669a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:27.877133 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-b87210fe-942f-4a96-99d7-bf068464669a None None] MFA Rules not processed for user `ff4736f3d50746b0bf5d6a917b2ca0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:36:27.878458 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-b87210fe-942f-4a96-99d7-bf068464669a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:36:27.970707 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1145/2311] 10.222.0.18 () {58 vars in 1009 bytes} [Wed Nov 2 07:36:27 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:36:27.983051 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab525dcf-721a-43a1-8c21-327afec352a0 service nova] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:36:27.984600 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-ab525dcf-721a-43a1-8c21-327afec352a0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:36:28.143927 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab525dcf-721a-43a1-8c21-327afec352a0 service nova] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:36:28.150091 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac160ede-a801-485e-ab18-a8048989b66e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9116f47889f4db39e0203d6cdae1593', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:36:28.151571 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac160ede-a801-485e-ab18-a8048989b66e service swift] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:28.152041 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac160ede-a801-485e-ab18-a8048989b66e service swift] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:28.152705 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac160ede-a801-485e-ab18-a8048989b66e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:28.170702 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac160ede-a801-485e-ab18-a8048989b66e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:36:28.173900 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac160ede-a801-485e-ab18-a8048989b66e service swift] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:36:28.211379 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1167/2312] 10.222.0.18 () {60 vars in 1384 bytes} [Wed Nov 2 07:36:27 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Nov 02 07:36:32.446499 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb5f8cc-20a4-4c59-891b-1d7b08a702d5 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:32.447815 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb5f8cc-20a4-4c59-891b-1d7b08a702d5 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:32.448543 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb5f8cc-20a4-4c59-891b-1d7b08a702d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:32.488858 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-9eb5f8cc-20a4-4c59-891b-1d7b08a702d5 None None] MFA Rules not processed for user `33c63dc278ac4eb79d2a4553fcbeda4f`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:36:32.490482 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-9eb5f8cc-20a4-4c59-891b-1d7b08a702d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:36:32.519969 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1146/2313] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:36:32 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:36:34.790585 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac160ede-a801-485e-ab18-a8048989b66e service swift] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:36:34.792082 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-ac160ede-a801-485e-ab18-a8048989b66e service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:36:34.805154 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac160ede-a801-485e-ab18-a8048989b66e service swift] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:36:34.808774 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1197af-c5c6-4e05-aedc-49b90b180dd6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3ec6ecb85f94ef59762877d9ea33ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:36:34.809367 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1197af-c5c6-4e05-aedc-49b90b180dd6 service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:34.809641 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1197af-c5c6-4e05-aedc-49b90b180dd6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:34.809908 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1197af-c5c6-4e05-aedc-49b90b180dd6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:34.811592 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-2c1197af-c5c6-4e05-aedc-49b90b180dd6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:36:34.825005 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1197af-c5c6-4e05-aedc-49b90b180dd6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:36:34.826578 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1197af-c5c6-4e05-aedc-49b90b180dd6 service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:36:34.850712 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1168/2314] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:36:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:36:38.676679 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb5f8cc-20a4-4c59-891b-1d7b08a702d5 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:36:38.678129 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-9eb5f8cc-20a4-4c59-891b-1d7b08a702d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:36:38.699602 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb5f8cc-20a4-4c59-891b-1d7b08a702d5 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:36:38.704607 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b6eb81-5a12-47a0-a9a4-547a450cea85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:36:38.705240 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b6eb81-5a12-47a0-a9a4-547a450cea85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:38.705510 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b6eb81-5a12-47a0-a9a4-547a450cea85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:38.705784 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b6eb81-5a12-47a0-a9a4-547a450cea85 admin admin] PATH_INFO: `/v3/users/47965dc7f5964bbfa4efff0822530965` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:38.707121 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b6eb81-5a12-47a0-a9a4-547a450cea85 admin admin] RBAC: Authorizing `identity:delete_user(user_id=47965dc7f5964bbfa4efff0822530965)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:36:38.709219 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b6eb81-5a12-47a0-a9a4-547a450cea85 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:36:38.760636 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-81b6eb81-5a12-47a0-a9a4-547a450cea85 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '47965dc7f5964bbfa4efff0822530965'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:36:38.768919 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-81b6eb81-5a12-47a0-a9a4-547a450cea85 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '47965dc7f5964bbfa4efff0822530965'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:36:38.769596 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-81b6eb81-5a12-47a0-a9a4-547a450cea85 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '47965dc7f5964bbfa4efff0822530965'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:36:38.777193 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-81b6eb81-5a12-47a0-a9a4-547a450cea85 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '47965dc7f5964bbfa4efff0822530965'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:36:38.791415 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-81b6eb81-5a12-47a0-a9a4-547a450cea85 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '47965dc7f5964bbfa4efff0822530965'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:36:38.798615 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1147/2315] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Nov 2 07:36:38 2022] DELETE /identity/v3/users/47965dc7f5964bbfa4efff0822530965 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:36:39.020524 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1197af-c5c6-4e05-aedc-49b90b180dd6 service neutron] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:36:39.040807 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1197af-c5c6-4e05-aedc-49b90b180dd6 service neutron] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:36:39.113927 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c220fd87-a9a6-4a1e-bd8a-66ae92260b10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:36:39.114635 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c220fd87-a9a6-4a1e-bd8a-66ae92260b10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:39.114910 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c220fd87-a9a6-4a1e-bd8a-66ae92260b10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:39.115174 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c220fd87-a9a6-4a1e-bd8a-66ae92260b10 admin admin] PATH_INFO: `/v3/projects/ba3bab98d31d49098a8e7242d51be646` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:39.116274 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c220fd87-a9a6-4a1e-bd8a-66ae92260b10 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ba3bab98d31d49098a8e7242d51be646)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:36:39.118266 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c220fd87-a9a6-4a1e-bd8a-66ae92260b10 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:36:39.145845 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:36:39.189968 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-c220fd87-a9a6-4a1e-bd8a-66ae92260b10 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ba3bab98d31d49098a8e7242d51be646', 'request_id': 'req-c220fd87-a9a6-4a1e-bd8a-66ae92260b10'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:36:39.198150 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-c220fd87-a9a6-4a1e-bd8a-66ae92260b10 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ba3bab98d31d49098a8e7242d51be646', 'request_id': 'req-c220fd87-a9a6-4a1e-bd8a-66ae92260b10'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:36:39.206902 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-c220fd87-a9a6-4a1e-bd8a-66ae92260b10 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ba3bab98d31d49098a8e7242d51be646', 'request_id': 'req-c220fd87-a9a6-4a1e-bd8a-66ae92260b10'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:36:39.220405 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-c220fd87-a9a6-4a1e-bd8a-66ae92260b10 admin admin] The token cache is being invalidate because project ba3bab98d31d49098a8e7242d51be646 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:36:39.220688 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-c220fd87-a9a6-4a1e-bd8a-66ae92260b10 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:36:39.221315 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-c220fd87-a9a6-4a1e-bd8a-66ae92260b10 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:36:39.222948 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1169/2316] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Nov 2 07:36:39 2022] DELETE /identity/v3/projects/ba3bab98d31d49098a8e7242d51be646 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:36:39.227926 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e126aec-0191-4f05-8798-9b67ffdc9f30 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:39.228228 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e126aec-0191-4f05-8798-9b67ffdc9f30 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:39.228538 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e126aec-0191-4f05-8798-9b67ffdc9f30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:39.248420 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-7e126aec-0191-4f05-8798-9b67ffdc9f30 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:36:39.249598 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-7e126aec-0191-4f05-8798-9b67ffdc9f30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:36:39.311828 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1148/2317] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:36:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:36:39.320033 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e973c11-24c6-4bf3-ad92-38444357ec80 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:39.320411 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e973c11-24c6-4bf3-ad92-38444357ec80 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:39.320744 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e973c11-24c6-4bf3-ad92-38444357ec80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:39.343705 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-3e973c11-24c6-4bf3-ad92-38444357ec80 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:36:39.345468 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-3e973c11-24c6-4bf3-ad92-38444357ec80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:36:39.354345 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1170/2318] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:36:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:36:39.358542 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e126aec-0191-4f05-8798-9b67ffdc9f30 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:36:39.369078 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e126aec-0191-4f05-8798-9b67ffdc9f30 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:36:39.373629 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c4e08e-56bc-4ac0-8c17-cd0c9bc418bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:36:39.374222 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c4e08e-56bc-4ac0-8c17-cd0c9bc418bd admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:39.374540 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c4e08e-56bc-4ac0-8c17-cd0c9bc418bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:39.374813 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c4e08e-56bc-4ac0-8c17-cd0c9bc418bd admin admin] PATH_INFO: `/v3/domains` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:39.375489 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c4e08e-56bc-4ac0-8c17-cd0c9bc418bd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:36:39.375782 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c4e08e-56bc-4ac0-8c17-cd0c9bc418bd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:36:39.377358 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c4e08e-56bc-4ac0-8c17-cd0c9bc418bd admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:36:39.384258 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:36:39.394197 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1149/2319] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Nov 2 07:36:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:36:39.395313 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e973c11-24c6-4bf3-ad92-38444357ec80 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:36:39.404492 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e973c11-24c6-4bf3-ad92-38444357ec80 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:36:39.409821 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5bad53-5b93-4111-9114-62fa1165c0ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:36:39.410447 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5bad53-5b93-4111-9114-62fa1165c0ed admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:39.410727 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5bad53-5b93-4111-9114-62fa1165c0ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:39.410996 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5bad53-5b93-4111-9114-62fa1165c0ed admin admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:39.411736 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5bad53-5b93-4111-9114-62fa1165c0ed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:36:39.413630 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5bad53-5b93-4111-9114-62fa1165c0ed admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:36:39.444567 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1171/2320] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Nov 2 07:36:39 2022] POST /identity/v3/projects => generated 379 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 07:36:39.449033 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c4e08e-56bc-4ac0-8c17-cd0c9bc418bd admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:36:39.461016 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c4e08e-56bc-4ac0-8c17-cd0c9bc418bd admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:36:39.529602 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2c3912-00d5-413f-9a70-badf4d1125c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:36:39.530226 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2c3912-00d5-413f-9a70-badf4d1125c0 admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:39.530545 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2c3912-00d5-413f-9a70-badf4d1125c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:39.530810 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2c3912-00d5-413f-9a70-badf4d1125c0 admin admin] PATH_INFO: `/v3/users` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:39.531439 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2c3912-00d5-413f-9a70-badf4d1125c0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:36:39.533970 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2c3912-00d5-413f-9a70-badf4d1125c0 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:36:39.561550 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1150/2321] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Nov 2 07:36:39 2022] POST /identity/v3/users => generated 290 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:36:39.566044 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5bad53-5b93-4111-9114-62fa1165c0ed admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:36:39.576359 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5bad53-5b93-4111-9114-62fa1165c0ed admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:36:39.582083 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5860f01-7c05-49f6-9834-661883d4dc18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:36:39.582712 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5860f01-7c05-49f6-9834-661883d4dc18 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:39.583024 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5860f01-7c05-49f6-9834-661883d4dc18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:39.583426 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5860f01-7c05-49f6-9834-661883d4dc18 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:39.584068 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5860f01-7c05-49f6-9834-661883d4dc18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:36:39.585652 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5860f01-7c05-49f6-9834-661883d4dc18 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:36:39.596408 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1172/2322] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:36:39 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:36:39.600270 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2c3912-00d5-413f-9a70-badf4d1125c0 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:36:39.610818 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2c3912-00d5-413f-9a70-badf4d1125c0 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:36:39.615430 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436998c9-78f7-44be-aa06-2439d1ddc340 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:36:39.616016 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436998c9-78f7-44be-aa06-2439d1ddc340 admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:39.616295 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436998c9-78f7-44be-aa06-2439d1ddc340 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:39.616567 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436998c9-78f7-44be-aa06-2439d1ddc340 admin admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:39.617201 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436998c9-78f7-44be-aa06-2439d1ddc340 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:36:39.618821 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436998c9-78f7-44be-aa06-2439d1ddc340 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:36:39.629874 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1151/2323] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:36:39 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:36:39.634718 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5860f01-7c05-49f6-9834-661883d4dc18 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:36:39.645699 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5860f01-7c05-49f6-9834-661883d4dc18 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:36:39.650227 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8001b0f-6bfb-4c23-99a7-8970654e02ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:36:39.650913 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8001b0f-6bfb-4c23-99a7-8970654e02ce admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:39.651188 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8001b0f-6bfb-4c23-99a7-8970654e02ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:39.651485 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8001b0f-6bfb-4c23-99a7-8970654e02ce admin admin] PATH_INFO: `/v3/projects/8a0d551fa66c40cabd1e84a892aac94c/users/aab7f664983248408807b373c4b17dcd/roles/b494fedd00f24f0b84a529e8672a156a` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:39.665907 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8001b0f-6bfb-4c23-99a7-8970654e02ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a0d551fa66c40cabd1e84a892aac94c, user_id=aab7f664983248408807b373c4b17dcd, role_id=b494fedd00f24f0b84a529e8672a156a)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:36:39.668687 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8001b0f-6bfb-4c23-99a7-8970654e02ce admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:36:39.681748 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1173/2324] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Nov 2 07:36:39 2022] PUT /identity/v3/projects/8a0d551fa66c40cabd1e84a892aac94c/users/aab7f664983248408807b373c4b17dcd/roles/b494fedd00f24f0b84a529e8672a156a => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 07:36:39.693889 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b125831a-6f75-4d9d-89e6-c7add1248b39 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:39.694419 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b125831a-6f75-4d9d-89e6-c7add1248b39 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:39.694528 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b125831a-6f75-4d9d-89e6-c7add1248b39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:39.723720 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-b125831a-6f75-4d9d-89e6-c7add1248b39 None None] MFA Rules not processed for user `aab7f664983248408807b373c4b17dcd`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:36:39.724933 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-b125831a-6f75-4d9d-89e6-c7add1248b39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:36:39.814481 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1152/2325] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:36:39 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:36:39.815379 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8001b0f-6bfb-4c23-99a7-8970654e02ce admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:36:39.892543 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8001b0f-6bfb-4c23-99a7-8970654e02ce admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:36:39.898311 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd444ee9-7009-4cc5-a722-bf7f0a8f2be7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:36:39.899369 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd444ee9-7009-4cc5-a722-bf7f0a8f2be7 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:39.899678 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd444ee9-7009-4cc5-a722-bf7f0a8f2be7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:39.899970 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd444ee9-7009-4cc5-a722-bf7f0a8f2be7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:39.900603 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd444ee9-7009-4cc5-a722-bf7f0a8f2be7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:36:39.902532 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd444ee9-7009-4cc5-a722-bf7f0a8f2be7 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:36:39.948807 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1174/2326] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Nov 2 07:36:39 2022] POST /identity/v3/projects => generated 381 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:36:39.949818 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b125831a-6f75-4d9d-89e6-c7add1248b39 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:36:39.961160 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b125831a-6f75-4d9d-89e6-c7add1248b39 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:36:40.029403 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca57736d-b44b-4d13-94b6-417f38446d86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:36:40.029980 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca57736d-b44b-4d13-94b6-417f38446d86 admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:40.030265 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca57736d-b44b-4d13-94b6-417f38446d86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:40.030681 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca57736d-b44b-4d13-94b6-417f38446d86 admin admin] PATH_INFO: `/v3/users` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:40.031565 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca57736d-b44b-4d13-94b6-417f38446d86 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:36:40.033584 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca57736d-b44b-4d13-94b6-417f38446d86 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:36:40.045185 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:36:40.055989 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1153/2327] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Nov 2 07:36:39 2022] POST /identity/v3/users => generated 297 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:36:40.060117 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd444ee9-7009-4cc5-a722-bf7f0a8f2be7 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:36:40.071261 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd444ee9-7009-4cc5-a722-bf7f0a8f2be7 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:36:40.076130 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02111a5-38e0-424b-bce0-c26c03339a98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:36:40.076932 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02111a5-38e0-424b-bce0-c26c03339a98 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:40.077216 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02111a5-38e0-424b-bce0-c26c03339a98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:40.077517 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02111a5-38e0-424b-bce0-c26c03339a98 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:40.078151 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02111a5-38e0-424b-bce0-c26c03339a98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:36:40.079886 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02111a5-38e0-424b-bce0-c26c03339a98 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:36:40.093771 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1175/2328] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:36:40 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:36:40.095755 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca57736d-b44b-4d13-94b6-417f38446d86 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:36:40.106015 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca57736d-b44b-4d13-94b6-417f38446d86 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:36:40.111519 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af04b1a-8601-43c0-88d7-13bf7184c905 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:36:40.112172 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af04b1a-8601-43c0-88d7-13bf7184c905 admin admin] REQUEST_METHOD: `PUT` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:40.112450 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af04b1a-8601-43c0-88d7-13bf7184c905 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:40.112719 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af04b1a-8601-43c0-88d7-13bf7184c905 admin admin] PATH_INFO: `/v3/projects/b65f485e9c4f4f498704dcd6d1aab234/users/a6f1ae9f976a474f835b18dd9f0178aa/roles/c4bf019fc6c44c71a6c984f6d0bab75f` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:40.127908 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af04b1a-8601-43c0-88d7-13bf7184c905 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b65f485e9c4f4f498704dcd6d1aab234, user_id=a6f1ae9f976a474f835b18dd9f0178aa, role_id=c4bf019fc6c44c71a6c984f6d0bab75f)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:36:40.130567 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af04b1a-8601-43c0-88d7-13bf7184c905 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:36:40.147184 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1154/2329] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Nov 2 07:36:40 2022] PUT /identity/v3/projects/b65f485e9c4f4f498704dcd6d1aab234/users/a6f1ae9f976a474f835b18dd9f0178aa/roles/c4bf019fc6c44c71a6c984f6d0bab75f => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:36:40.151496 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02111a5-38e0-424b-bce0-c26c03339a98 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:36:40.161798 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02111a5-38e0-424b-bce0-c26c03339a98 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:36:40.176594 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:36:40.224990 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5742171d-53c9-4011-9808-3e1d704b3bc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:36:40.225590 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5742171d-53c9-4011-9808-3e1d704b3bc3 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:40.225864 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5742171d-53c9-4011-9808-3e1d704b3bc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:40.226133 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5742171d-53c9-4011-9808-3e1d704b3bc3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:40.226794 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5742171d-53c9-4011-9808-3e1d704b3bc3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:36:40.228363 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5742171d-53c9-4011-9808-3e1d704b3bc3 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:36:40.238954 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1176/2330] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:36:40 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:36:40.243122 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af04b1a-8601-43c0-88d7-13bf7184c905 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:36:40.254529 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af04b1a-8601-43c0-88d7-13bf7184c905 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:36:40.259620 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c58a07f-cd25-4588-a749-f5ae3cceb84f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:36:40.260292 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c58a07f-cd25-4588-a749-f5ae3cceb84f admin admin] REQUEST_METHOD: `PUT` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:40.260596 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c58a07f-cd25-4588-a749-f5ae3cceb84f admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:40.260895 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c58a07f-cd25-4588-a749-f5ae3cceb84f admin admin] PATH_INFO: `/v3/projects/b65f485e9c4f4f498704dcd6d1aab234/users/a6f1ae9f976a474f835b18dd9f0178aa/roles/c4bf019fc6c44c71a6c984f6d0bab75f` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:40.262795 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c58a07f-cd25-4588-a749-f5ae3cceb84f admin admin] RBAC: Authorizing `identity:create_grant(project_id=b65f485e9c4f4f498704dcd6d1aab234, user_id=a6f1ae9f976a474f835b18dd9f0178aa, role_id=c4bf019fc6c44c71a6c984f6d0bab75f)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:36:40.265410 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c58a07f-cd25-4588-a749-f5ae3cceb84f admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:36:40.276620 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1155/2331] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Nov 2 07:36:40 2022] PUT /identity/v3/projects/b65f485e9c4f4f498704dcd6d1aab234/users/a6f1ae9f976a474f835b18dd9f0178aa/roles/c4bf019fc6c44c71a6c984f6d0bab75f => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:36:40.287859 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0c754a-8237-4d83-93d4-ec0c7902bf7b None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:40.288145 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0c754a-8237-4d83-93d4-ec0c7902bf7b None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:40.288419 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0c754a-8237-4d83-93d4-ec0c7902bf7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:40.317067 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-ad0c754a-8237-4d83-93d4-ec0c7902bf7b None None] MFA Rules not processed for user `a6f1ae9f976a474f835b18dd9f0178aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:36:40.318296 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-ad0c754a-8237-4d83-93d4-ec0c7902bf7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:36:40.393470 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1177/2332] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:36:40 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:36:40.404982 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c58a07f-cd25-4588-a749-f5ae3cceb84f admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:36:40.407072 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-8c58a07f-cd25-4588-a749-f5ae3cceb84f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:36:40.471455 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c58a07f-cd25-4588-a749-f5ae3cceb84f admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:36:40.475277 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90de86f-2a5a-47fa-b626-435f083f030e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43bb326fb82e4d2c83fb2eaf91860dc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:36:40.475882 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90de86f-2a5a-47fa-b626-435f083f030e service cinder] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:40.476811 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90de86f-2a5a-47fa-b626-435f083f030e service cinder] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:40.476811 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90de86f-2a5a-47fa-b626-435f083f030e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:40.487662 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90de86f-2a5a-47fa-b626-435f083f030e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:36:40.489103 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90de86f-2a5a-47fa-b626-435f083f030e service cinder] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:36:40.498920 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1156/2333] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:36:40 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:36:40.590000 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0c754a-8237-4d83-93d4-ec0c7902bf7b None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:36:40.601440 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0c754a-8237-4d83-93d4-ec0c7902bf7b None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:36:40.616088 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b9c7af-fcef-4d9b-a96c-83d8b43a0213 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43bb326fb82e4d2c83fb2eaf91860dc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:36:40.616841 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b9c7af-fcef-4d9b-a96c-83d8b43a0213 service cinder] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:40.617212 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b9c7af-fcef-4d9b-a96c-83d8b43a0213 service cinder] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:40.617579 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b9c7af-fcef-4d9b-a96c-83d8b43a0213 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:40.688890 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b9c7af-fcef-4d9b-a96c-83d8b43a0213 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:36:40.690267 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b9c7af-fcef-4d9b-a96c-83d8b43a0213 service cinder] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:36:40.705145 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1178/2334] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:36:40 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:36:50.495052 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=43,keystone:INSERT=2 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:36:50.701875 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=83 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:36:50.739277 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90de86f-2a5a-47fa-b626-435f083f030e service cinder] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:36:50.841753 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90de86f-2a5a-47fa-b626-435f083f030e service cinder] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:36:50.848257 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac729688-3e5f-48c6-8e2f-1b2bd58fb5e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:36:50.848898 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac729688-3e5f-48c6-8e2f-1b2bd58fb5e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:50.849216 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac729688-3e5f-48c6-8e2f-1b2bd58fb5e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:50.849485 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac729688-3e5f-48c6-8e2f-1b2bd58fb5e5 admin admin] PATH_INFO: `/v3/users/aab7f664983248408807b373c4b17dcd` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:50.850648 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac729688-3e5f-48c6-8e2f-1b2bd58fb5e5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aab7f664983248408807b373c4b17dcd)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:36:50.853154 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac729688-3e5f-48c6-8e2f-1b2bd58fb5e5 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:36:50.916709 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-ac729688-3e5f-48c6-8e2f-1b2bd58fb5e5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aab7f664983248408807b373c4b17dcd'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:36:50.925765 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-ac729688-3e5f-48c6-8e2f-1b2bd58fb5e5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aab7f664983248408807b373c4b17dcd'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:36:50.926534 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-ac729688-3e5f-48c6-8e2f-1b2bd58fb5e5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aab7f664983248408807b373c4b17dcd'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:36:50.935533 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-ac729688-3e5f-48c6-8e2f-1b2bd58fb5e5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aab7f664983248408807b373c4b17dcd'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:36:50.951483 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-ac729688-3e5f-48c6-8e2f-1b2bd58fb5e5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aab7f664983248408807b373c4b17dcd'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:36:50.958113 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1157/2335] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Nov 2 07:36:50 2022] DELETE /identity/v3/users/aab7f664983248408807b373c4b17dcd => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:36:50.962714 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c03c1e-e420-4dc2-ac65-dce726b9afbb None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:50.963076 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c03c1e-e420-4dc2-ac65-dce726b9afbb None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:50.963422 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c03c1e-e420-4dc2-ac65-dce726b9afbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:50.984617 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-15c03c1e-e420-4dc2-ac65-dce726b9afbb None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:36:50.986049 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-15c03c1e-e420-4dc2-ac65-dce726b9afbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:36:51.047743 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1179/2336] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:36:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:36:51.057250 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac729688-3e5f-48c6-8e2f-1b2bd58fb5e5 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:36:51.058684 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-ac729688-3e5f-48c6-8e2f-1b2bd58fb5e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:36:51.120793 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac729688-3e5f-48c6-8e2f-1b2bd58fb5e5 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:36:51.124566 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2dbb772-caca-49c8-bebd-009fa85ffc43 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3ec6ecb85f94ef59762877d9ea33ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:36:51.125175 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dbb772-caca-49c8-bebd-009fa85ffc43 service neutron] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:51.125457 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dbb772-caca-49c8-bebd-009fa85ffc43 service neutron] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:51.126444 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dbb772-caca-49c8-bebd-009fa85ffc43 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:51.138096 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2dbb772-caca-49c8-bebd-009fa85ffc43 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:36:51.139613 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2dbb772-caca-49c8-bebd-009fa85ffc43 service neutron] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:36:51.152098 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1158/2337] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:36:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:36:51.465701 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c03c1e-e420-4dc2-ac65-dce726b9afbb None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:36:51.467267 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-15c03c1e-e420-4dc2-ac65-dce726b9afbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:36:51.480379 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c03c1e-e420-4dc2-ac65-dce726b9afbb None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:36:51.484979 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ced742-393c-42d6-9946-c11ca90a2d6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:36:51.485654 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ced742-393c-42d6-9946-c11ca90a2d6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:51.485931 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ced742-393c-42d6-9946-c11ca90a2d6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:51.486205 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ced742-393c-42d6-9946-c11ca90a2d6a admin admin] PATH_INFO: `/v3/projects/8a0d551fa66c40cabd1e84a892aac94c` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:51.487373 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ced742-393c-42d6-9946-c11ca90a2d6a admin admin] RBAC: Authorizing `identity:delete_project(project_id=8a0d551fa66c40cabd1e84a892aac94c)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:36:51.489265 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ced742-393c-42d6-9946-c11ca90a2d6a admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:36:51.537447 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-67ced742-393c-42d6-9946-c11ca90a2d6a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8a0d551fa66c40cabd1e84a892aac94c', 'request_id': 'req-67ced742-393c-42d6-9946-c11ca90a2d6a'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:36:51.543546 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-67ced742-393c-42d6-9946-c11ca90a2d6a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8a0d551fa66c40cabd1e84a892aac94c', 'request_id': 'req-67ced742-393c-42d6-9946-c11ca90a2d6a'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:36:51.549616 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-67ced742-393c-42d6-9946-c11ca90a2d6a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8a0d551fa66c40cabd1e84a892aac94c', 'request_id': 'req-67ced742-393c-42d6-9946-c11ca90a2d6a'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:36:51.562646 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-67ced742-393c-42d6-9946-c11ca90a2d6a admin admin] The token cache is being invalidate because project 8a0d551fa66c40cabd1e84a892aac94c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:36:51.562927 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-67ced742-393c-42d6-9946-c11ca90a2d6a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:36:51.563739 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-67ced742-393c-42d6-9946-c11ca90a2d6a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:36:51.565213 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1180/2338] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Nov 2 07:36:51 2022] DELETE /identity/v3/projects/8a0d551fa66c40cabd1e84a892aac94c => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:36:51.569409 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2dbb772-caca-49c8-bebd-009fa85ffc43 service neutron] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:36:51.579782 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2dbb772-caca-49c8-bebd-009fa85ffc43 service neutron] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:36:51.598568 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:36:51.637458 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43b17b5-1c4b-4758-9e0c-dd67a8aa3562 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:36:51.638083 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43b17b5-1c4b-4758-9e0c-dd67a8aa3562 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:51.638382 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43b17b5-1c4b-4758-9e0c-dd67a8aa3562 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:51.638672 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43b17b5-1c4b-4758-9e0c-dd67a8aa3562 admin admin] PATH_INFO: `/v3/users/a6f1ae9f976a474f835b18dd9f0178aa` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:51.639816 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43b17b5-1c4b-4758-9e0c-dd67a8aa3562 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a6f1ae9f976a474f835b18dd9f0178aa)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:36:51.642445 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43b17b5-1c4b-4758-9e0c-dd67a8aa3562 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:36:51.692128 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-b43b17b5-1c4b-4758-9e0c-dd67a8aa3562 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a6f1ae9f976a474f835b18dd9f0178aa'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:36:51.699820 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-b43b17b5-1c4b-4758-9e0c-dd67a8aa3562 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a6f1ae9f976a474f835b18dd9f0178aa'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:36:51.700714 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-b43b17b5-1c4b-4758-9e0c-dd67a8aa3562 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a6f1ae9f976a474f835b18dd9f0178aa'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:36:51.708968 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-b43b17b5-1c4b-4758-9e0c-dd67a8aa3562 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a6f1ae9f976a474f835b18dd9f0178aa'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:36:51.722623 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-b43b17b5-1c4b-4758-9e0c-dd67a8aa3562 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a6f1ae9f976a474f835b18dd9f0178aa'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:36:51.731607 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1159/2339] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Nov 2 07:36:51 2022] DELETE /identity/v3/users/a6f1ae9f976a474f835b18dd9f0178aa => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:36:52.074606 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ced742-393c-42d6-9946-c11ca90a2d6a admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:36:52.076087 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-67ced742-393c-42d6-9946-c11ca90a2d6a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:36:52.090812 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ced742-393c-42d6-9946-c11ca90a2d6a admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:36:52.159695 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88731ae-40ac-444f-aef6-c0ab43f5f3b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:36:52.160334 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88731ae-40ac-444f-aef6-c0ab43f5f3b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:52.160614 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88731ae-40ac-444f-aef6-c0ab43f5f3b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:52.160883 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88731ae-40ac-444f-aef6-c0ab43f5f3b7 admin admin] PATH_INFO: `/v3/projects/b65f485e9c4f4f498704dcd6d1aab234` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:52.161981 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88731ae-40ac-444f-aef6-c0ab43f5f3b7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b65f485e9c4f4f498704dcd6d1aab234)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:36:52.163901 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88731ae-40ac-444f-aef6-c0ab43f5f3b7 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:36:52.194557 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:36:52.213574 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-a88731ae-40ac-444f-aef6-c0ab43f5f3b7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b65f485e9c4f4f498704dcd6d1aab234', 'request_id': 'req-a88731ae-40ac-444f-aef6-c0ab43f5f3b7'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:36:52.220847 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-a88731ae-40ac-444f-aef6-c0ab43f5f3b7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b65f485e9c4f4f498704dcd6d1aab234', 'request_id': 'req-a88731ae-40ac-444f-aef6-c0ab43f5f3b7'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:36:52.227302 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-a88731ae-40ac-444f-aef6-c0ab43f5f3b7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b65f485e9c4f4f498704dcd6d1aab234', 'request_id': 'req-a88731ae-40ac-444f-aef6-c0ab43f5f3b7'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:36:52.242550 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-a88731ae-40ac-444f-aef6-c0ab43f5f3b7 admin admin] The token cache is being invalidate because project b65f485e9c4f4f498704dcd6d1aab234 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:36:52.242856 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-a88731ae-40ac-444f-aef6-c0ab43f5f3b7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:36:52.243479 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-a88731ae-40ac-444f-aef6-c0ab43f5f3b7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:36:52.244999 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1181/2340] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Nov 2 07:36:52 2022] DELETE /identity/v3/projects/b65f485e9c4f4f498704dcd6d1aab234 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:36:52.249854 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef96948e-8560-4a6b-af02-cf1c9901b647 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:52.250140 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef96948e-8560-4a6b-af02-cf1c9901b647 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:52.250457 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef96948e-8560-4a6b-af02-cf1c9901b647 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:52.275860 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-ef96948e-8560-4a6b-af02-cf1c9901b647 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:36:52.277048 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-ef96948e-8560-4a6b-af02-cf1c9901b647 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:36:52.336615 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1160/2341] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:36:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:36:52.345428 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2aa0c2-c6e7-4690-9bbe-17e813a5d387 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:52.345739 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2aa0c2-c6e7-4690-9bbe-17e813a5d387 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:52.345985 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2aa0c2-c6e7-4690-9bbe-17e813a5d387 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:52.365485 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-ce2aa0c2-c6e7-4690-9bbe-17e813a5d387 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:36:52.366749 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-ce2aa0c2-c6e7-4690-9bbe-17e813a5d387 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:36:52.374586 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1182/2342] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:36:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:36:52.378836 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef96948e-8560-4a6b-af02-cf1c9901b647 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:36:52.388571 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef96948e-8560-4a6b-af02-cf1c9901b647 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:36:52.393689 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12243a09-0a5b-4fa8-aeb1-051f5cada545 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:36:52.394279 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12243a09-0a5b-4fa8-aeb1-051f5cada545 admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:52.394614 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12243a09-0a5b-4fa8-aeb1-051f5cada545 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:52.394886 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12243a09-0a5b-4fa8-aeb1-051f5cada545 admin admin] PATH_INFO: `/v3/domains` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:52.395522 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12243a09-0a5b-4fa8-aeb1-051f5cada545 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:36:52.395824 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12243a09-0a5b-4fa8-aeb1-051f5cada545 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:36:52.397403 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12243a09-0a5b-4fa8-aeb1-051f5cada545 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:36:52.409546 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1161/2343] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Nov 2 07:36:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:36:52.413726 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2aa0c2-c6e7-4690-9bbe-17e813a5d387 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:36:52.422980 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2aa0c2-c6e7-4690-9bbe-17e813a5d387 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:36:52.427613 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039c75be-1772-4a34-8314-ac398087fdb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:36:52.428199 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039c75be-1772-4a34-8314-ac398087fdb1 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:52.428476 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039c75be-1772-4a34-8314-ac398087fdb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:52.428747 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039c75be-1772-4a34-8314-ac398087fdb1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:52.429376 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039c75be-1772-4a34-8314-ac398087fdb1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:36:52.431317 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039c75be-1772-4a34-8314-ac398087fdb1 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:36:52.458339 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1183/2344] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Nov 2 07:36:52 2022] POST /identity/v3/projects => generated 387 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:36:52.462742 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12243a09-0a5b-4fa8-aeb1-051f5cada545 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:36:52.473325 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12243a09-0a5b-4fa8-aeb1-051f5cada545 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:36:52.499921 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:36:52.532828 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd8b93b-8f5a-4f4e-81ef-ca82f54a1b24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:36:52.533424 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd8b93b-8f5a-4f4e-81ef-ca82f54a1b24 admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:52.533729 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd8b93b-8f5a-4f4e-81ef-ca82f54a1b24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:52.533999 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd8b93b-8f5a-4f4e-81ef-ca82f54a1b24 admin admin] PATH_INFO: `/v3/users` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:52.534678 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd8b93b-8f5a-4f4e-81ef-ca82f54a1b24 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:36:52.536629 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd8b93b-8f5a-4f4e-81ef-ca82f54a1b24 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:36:52.556872 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1162/2345] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Nov 2 07:36:52 2022] POST /identity/v3/users => generated 294 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 07:36:52.561096 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039c75be-1772-4a34-8314-ac398087fdb1 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:36:52.571844 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039c75be-1772-4a34-8314-ac398087fdb1 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:36:52.576919 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf1dd07-5ac9-4bf7-bbc6-5cbddfecc416 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:36:52.577514 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf1dd07-5ac9-4bf7-bbc6-5cbddfecc416 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:52.577791 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf1dd07-5ac9-4bf7-bbc6-5cbddfecc416 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:52.578084 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf1dd07-5ac9-4bf7-bbc6-5cbddfecc416 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:52.578739 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf1dd07-5ac9-4bf7-bbc6-5cbddfecc416 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:36:52.581047 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf1dd07-5ac9-4bf7-bbc6-5cbddfecc416 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:36:52.591027 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1184/2346] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:36:52 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:36:52.594999 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd8b93b-8f5a-4f4e-81ef-ca82f54a1b24 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:36:52.604838 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd8b93b-8f5a-4f4e-81ef-ca82f54a1b24 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:36:52.609347 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4992980-9bbe-4cad-b59c-17f87c185c0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:36:52.609934 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4992980-9bbe-4cad-b59c-17f87c185c0d admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:52.610260 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4992980-9bbe-4cad-b59c-17f87c185c0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:52.610530 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4992980-9bbe-4cad-b59c-17f87c185c0d admin admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:52.611144 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4992980-9bbe-4cad-b59c-17f87c185c0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:36:52.612723 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4992980-9bbe-4cad-b59c-17f87c185c0d admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:36:52.623209 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1163/2347] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:36:52 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:36:52.627743 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf1dd07-5ac9-4bf7-bbc6-5cbddfecc416 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:36:52.637814 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf1dd07-5ac9-4bf7-bbc6-5cbddfecc416 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:36:52.650101 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b29214a-a889-4684-8742-8c4796e6b586 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:36:52.650344 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b29214a-a889-4684-8742-8c4796e6b586 admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:52.650592 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b29214a-a889-4684-8742-8c4796e6b586 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:52.650869 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b29214a-a889-4684-8742-8c4796e6b586 admin admin] PATH_INFO: `/v3/projects/064246b8004c49adb4e718d95eca6903/users/4b35184f551d45249ea529135a57007f/roles/b494fedd00f24f0b84a529e8672a156a` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:52.666753 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b29214a-a889-4684-8742-8c4796e6b586 admin admin] RBAC: Authorizing `identity:create_grant(project_id=064246b8004c49adb4e718d95eca6903, user_id=4b35184f551d45249ea529135a57007f, role_id=b494fedd00f24f0b84a529e8672a156a)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:36:52.669443 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b29214a-a889-4684-8742-8c4796e6b586 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:36:52.682113 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1185/2348] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Nov 2 07:36:52 2022] PUT /identity/v3/projects/064246b8004c49adb4e718d95eca6903/users/4b35184f551d45249ea529135a57007f/roles/b494fedd00f24f0b84a529e8672a156a => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 07:36:52.694222 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a630e2-7cce-481d-8dd6-d6b185cbd1bd None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:52.694534 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a630e2-7cce-481d-8dd6-d6b185cbd1bd None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:52.694809 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a630e2-7cce-481d-8dd6-d6b185cbd1bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:52.723270 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-e0a630e2-7cce-481d-8dd6-d6b185cbd1bd None None] MFA Rules not processed for user `4b35184f551d45249ea529135a57007f`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:36:52.724471 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-e0a630e2-7cce-481d-8dd6-d6b185cbd1bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:36:52.796905 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1164/2349] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:36:52 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:36:52.802024 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b29214a-a889-4684-8742-8c4796e6b586 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:36:52.879326 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b29214a-a889-4684-8742-8c4796e6b586 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:36:52.884287 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef2ab67-2fa7-47c4-ba7b-2fb968621b95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:36:52.884891 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef2ab67-2fa7-47c4-ba7b-2fb968621b95 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:52.885179 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef2ab67-2fa7-47c4-ba7b-2fb968621b95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:52.885452 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef2ab67-2fa7-47c4-ba7b-2fb968621b95 admin admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:52.886076 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef2ab67-2fa7-47c4-ba7b-2fb968621b95 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:36:52.888012 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef2ab67-2fa7-47c4-ba7b-2fb968621b95 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:36:52.918828 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1186/2350] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Nov 2 07:36:52 2022] POST /identity/v3/projects => generated 389 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:36:52.919936 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a630e2-7cce-481d-8dd6-d6b185cbd1bd None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:36:52.931816 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a630e2-7cce-481d-8dd6-d6b185cbd1bd None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:36:52.993624 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d57ccc0-e6db-4aad-84a8-ff0020dbf9e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:36:52.994220 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d57ccc0-e6db-4aad-84a8-ff0020dbf9e2 admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:52.994523 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d57ccc0-e6db-4aad-84a8-ff0020dbf9e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:52.994791 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d57ccc0-e6db-4aad-84a8-ff0020dbf9e2 admin admin] PATH_INFO: `/v3/users` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:52.995403 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d57ccc0-e6db-4aad-84a8-ff0020dbf9e2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:36:52.997283 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d57ccc0-e6db-4aad-84a8-ff0020dbf9e2 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:36:53.019100 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1165/2351] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Nov 2 07:36:52 2022] POST /identity/v3/users => generated 301 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:36:53.023382 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef2ab67-2fa7-47c4-ba7b-2fb968621b95 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:36:53.033515 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef2ab67-2fa7-47c4-ba7b-2fb968621b95 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:36:53.038470 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe3f000-fe22-4614-8e81-0a471ee9192b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:36:53.039065 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe3f000-fe22-4614-8e81-0a471ee9192b admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:53.039343 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe3f000-fe22-4614-8e81-0a471ee9192b admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:53.039612 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe3f000-fe22-4614-8e81-0a471ee9192b admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:53.040245 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe3f000-fe22-4614-8e81-0a471ee9192b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:36:53.041804 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe3f000-fe22-4614-8e81-0a471ee9192b admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:36:53.052900 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1187/2352] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:36:53 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:36:53.057427 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d57ccc0-e6db-4aad-84a8-ff0020dbf9e2 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:36:53.068161 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d57ccc0-e6db-4aad-84a8-ff0020dbf9e2 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:36:53.072989 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09b8d4d-516d-4320-939c-36c66d7234df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:36:53.073620 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09b8d4d-516d-4320-939c-36c66d7234df admin admin] REQUEST_METHOD: `PUT` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:53.073900 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09b8d4d-516d-4320-939c-36c66d7234df admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:53.074203 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09b8d4d-516d-4320-939c-36c66d7234df admin admin] PATH_INFO: `/v3/projects/1f3a7e48d95a4f638bcef23697b811ee/users/ebd1e625eae34b77bc334aadbe806bf5/roles/c4bf019fc6c44c71a6c984f6d0bab75f` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:53.088483 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09b8d4d-516d-4320-939c-36c66d7234df admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f3a7e48d95a4f638bcef23697b811ee, user_id=ebd1e625eae34b77bc334aadbe806bf5, role_id=c4bf019fc6c44c71a6c984f6d0bab75f)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:36:53.091139 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09b8d4d-516d-4320-939c-36c66d7234df admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:36:53.103683 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1166/2353] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Nov 2 07:36:53 2022] PUT /identity/v3/projects/1f3a7e48d95a4f638bcef23697b811ee/users/ebd1e625eae34b77bc334aadbe806bf5/roles/c4bf019fc6c44c71a6c984f6d0bab75f => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:36:53.107662 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe3f000-fe22-4614-8e81-0a471ee9192b admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:36:53.118575 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe3f000-fe22-4614-8e81-0a471ee9192b admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:36:53.139837 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:36:53.179049 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d053a5b1-7433-43aa-a4b5-cab94a611cde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:36:53.179564 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d053a5b1-7433-43aa-a4b5-cab94a611cde admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:53.179841 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d053a5b1-7433-43aa-a4b5-cab94a611cde admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:53.180111 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d053a5b1-7433-43aa-a4b5-cab94a611cde admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:53.180745 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d053a5b1-7433-43aa-a4b5-cab94a611cde admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:36:53.182319 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d053a5b1-7433-43aa-a4b5-cab94a611cde admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:36:53.193322 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1188/2354] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:36:53 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:36:53.197477 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09b8d4d-516d-4320-939c-36c66d7234df admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:36:53.202880 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:36:53.209601 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09b8d4d-516d-4320-939c-36c66d7234df admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:36:53.215344 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43738b59-9232-45a4-bfd0-42bb5e3647f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:36:53.216284 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43738b59-9232-45a4-bfd0-42bb5e3647f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:53.216571 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43738b59-9232-45a4-bfd0-42bb5e3647f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:53.216846 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43738b59-9232-45a4-bfd0-42bb5e3647f3 admin admin] PATH_INFO: `/v3/projects/1f3a7e48d95a4f638bcef23697b811ee/users/ebd1e625eae34b77bc334aadbe806bf5/roles/c4bf019fc6c44c71a6c984f6d0bab75f` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:53.218835 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43738b59-9232-45a4-bfd0-42bb5e3647f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f3a7e48d95a4f638bcef23697b811ee, user_id=ebd1e625eae34b77bc334aadbe806bf5, role_id=c4bf019fc6c44c71a6c984f6d0bab75f)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:36:53.222086 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43738b59-9232-45a4-bfd0-42bb5e3647f3 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:36:53.232584 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1167/2355] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Nov 2 07:36:53 2022] PUT /identity/v3/projects/1f3a7e48d95a4f638bcef23697b811ee/users/ebd1e625eae34b77bc334aadbe806bf5/roles/c4bf019fc6c44c71a6c984f6d0bab75f => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:36:53.244348 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82547162-3521-4385-8bcb-d38dbfc17152 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:53.244704 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82547162-3521-4385-8bcb-d38dbfc17152 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:53.245046 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82547162-3521-4385-8bcb-d38dbfc17152 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:53.273252 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-82547162-3521-4385-8bcb-d38dbfc17152 None None] MFA Rules not processed for user `ebd1e625eae34b77bc334aadbe806bf5`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:36:53.274478 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-82547162-3521-4385-8bcb-d38dbfc17152 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:36:53.357541 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1189/2356] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:36:53 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:36:53.382897 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43738b59-9232-45a4-bfd0-42bb5e3647f3 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:36:53.385047 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-43738b59-9232-45a4-bfd0-42bb5e3647f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:36:53.446515 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43738b59-9232-45a4-bfd0-42bb5e3647f3 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:36:53.450204 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5523186-e155-414e-9ae0-52f646ce8276 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43bb326fb82e4d2c83fb2eaf91860dc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:36:53.450845 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5523186-e155-414e-9ae0-52f646ce8276 service cinder] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:53.451129 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5523186-e155-414e-9ae0-52f646ce8276 service cinder] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:53.451404 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5523186-e155-414e-9ae0-52f646ce8276 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:53.460961 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5523186-e155-414e-9ae0-52f646ce8276 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:36:53.462410 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5523186-e155-414e-9ae0-52f646ce8276 service cinder] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:36:53.473172 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1168/2357] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:36:53 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 07:36:53.615445 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82547162-3521-4385-8bcb-d38dbfc17152 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:36:53.685932 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82547162-3521-4385-8bcb-d38dbfc17152 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:36:53.691387 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b56b8e6-c095-418f-a9e9-f9c1b5b752ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:36:53.692016 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b56b8e6-c095-418f-a9e9-f9c1b5b752ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:53.692293 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b56b8e6-c095-418f-a9e9-f9c1b5b752ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:53.692563 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b56b8e6-c095-418f-a9e9-f9c1b5b752ea admin admin] PATH_INFO: `/v3/users/4b35184f551d45249ea529135a57007f` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:53.693676 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b56b8e6-c095-418f-a9e9-f9c1b5b752ea admin admin] RBAC: Authorizing `identity:delete_user(user_id=4b35184f551d45249ea529135a57007f)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:36:53.695598 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b56b8e6-c095-418f-a9e9-f9c1b5b752ea admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:36:53.744531 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-9b56b8e6-c095-418f-a9e9-f9c1b5b752ea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4b35184f551d45249ea529135a57007f'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:36:53.746324 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:36:53.753713 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-9b56b8e6-c095-418f-a9e9-f9c1b5b752ea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4b35184f551d45249ea529135a57007f'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:36:53.754430 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-9b56b8e6-c095-418f-a9e9-f9c1b5b752ea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4b35184f551d45249ea529135a57007f'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:36:53.761942 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-9b56b8e6-c095-418f-a9e9-f9c1b5b752ea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4b35184f551d45249ea529135a57007f'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:36:53.762623 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-9b56b8e6-c095-418f-a9e9-f9c1b5b752ea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4b35184f551d45249ea529135a57007f'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:36:53.783582 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1190/2358] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Nov 2 07:36:53 2022] DELETE /identity/v3/users/4b35184f551d45249ea529135a57007f => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 07:36:53.788195 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91bced6-d24c-4b29-bc38-972bf1ffacdd None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:53.788719 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91bced6-d24c-4b29-bc38-972bf1ffacdd None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:53.788817 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91bced6-d24c-4b29-bc38-972bf1ffacdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:53.807420 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-f91bced6-d24c-4b29-bc38-972bf1ffacdd None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:36:53.808279 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-f91bced6-d24c-4b29-bc38-972bf1ffacdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:36:53.869844 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1169/2359] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:36:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:36:53.879612 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b56b8e6-c095-418f-a9e9-f9c1b5b752ea admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:36:53.881074 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-9b56b8e6-c095-418f-a9e9-f9c1b5b752ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:36:53.940476 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b56b8e6-c095-418f-a9e9-f9c1b5b752ea admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:36:53.944147 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fef665-0f3b-457e-8644-f39f06ade6f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3ec6ecb85f94ef59762877d9ea33ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:36:53.944753 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fef665-0f3b-457e-8644-f39f06ade6f5 service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:53.945032 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fef665-0f3b-457e-8644-f39f06ade6f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:53.945301 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fef665-0f3b-457e-8644-f39f06ade6f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:53.954966 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fef665-0f3b-457e-8644-f39f06ade6f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:36:53.956382 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fef665-0f3b-457e-8644-f39f06ade6f5 service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:36:53.965936 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1191/2360] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:36:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:36:54.383706 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91bced6-d24c-4b29-bc38-972bf1ffacdd None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:36:54.385107 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-f91bced6-d24c-4b29-bc38-972bf1ffacdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:36:54.402534 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91bced6-d24c-4b29-bc38-972bf1ffacdd None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:36:54.408897 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99124575-3f9a-4e90-b8ce-0a791d69edfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:36:54.409622 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99124575-3f9a-4e90-b8ce-0a791d69edfa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:54.409964 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99124575-3f9a-4e90-b8ce-0a791d69edfa admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:54.410346 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99124575-3f9a-4e90-b8ce-0a791d69edfa admin admin] PATH_INFO: `/v3/projects/064246b8004c49adb4e718d95eca6903` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:54.412389 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99124575-3f9a-4e90-b8ce-0a791d69edfa admin admin] RBAC: Authorizing `identity:delete_project(project_id=064246b8004c49adb4e718d95eca6903)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:36:54.415643 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99124575-3f9a-4e90-b8ce-0a791d69edfa admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:36:54.481649 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-99124575-3f9a-4e90-b8ce-0a791d69edfa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '064246b8004c49adb4e718d95eca6903', 'request_id': 'req-99124575-3f9a-4e90-b8ce-0a791d69edfa'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:36:54.489665 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-99124575-3f9a-4e90-b8ce-0a791d69edfa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '064246b8004c49adb4e718d95eca6903', 'request_id': 'req-99124575-3f9a-4e90-b8ce-0a791d69edfa'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:36:54.497943 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-99124575-3f9a-4e90-b8ce-0a791d69edfa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '064246b8004c49adb4e718d95eca6903', 'request_id': 'req-99124575-3f9a-4e90-b8ce-0a791d69edfa'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:36:54.509370 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-99124575-3f9a-4e90-b8ce-0a791d69edfa admin admin] The token cache is being invalidate because project 064246b8004c49adb4e718d95eca6903 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81501) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:36:54.509661 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-99124575-3f9a-4e90-b8ce-0a791d69edfa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:36:54.510289 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-99124575-3f9a-4e90-b8ce-0a791d69edfa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:36:54.511814 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1170/2361] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Nov 2 07:36:54 2022] DELETE /identity/v3/projects/064246b8004c49adb4e718d95eca6903 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:36:54.515852 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fef665-0f3b-457e-8644-f39f06ade6f5 service neutron] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:36:54.529164 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fef665-0f3b-457e-8644-f39f06ade6f5 service neutron] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:36:54.593859 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693e7754-eca8-417b-bb8f-b921df19444b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:36:54.594718 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693e7754-eca8-417b-bb8f-b921df19444b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:54.594996 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693e7754-eca8-417b-bb8f-b921df19444b admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:54.595433 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693e7754-eca8-417b-bb8f-b921df19444b admin admin] PATH_INFO: `/v3/users/ebd1e625eae34b77bc334aadbe806bf5` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:54.596599 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693e7754-eca8-417b-bb8f-b921df19444b admin admin] RBAC: Authorizing `identity:delete_user(user_id=ebd1e625eae34b77bc334aadbe806bf5)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:36:54.598562 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693e7754-eca8-417b-bb8f-b921df19444b admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:36:54.660667 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-693e7754-eca8-417b-bb8f-b921df19444b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ebd1e625eae34b77bc334aadbe806bf5'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:36:54.669480 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-693e7754-eca8-417b-bb8f-b921df19444b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ebd1e625eae34b77bc334aadbe806bf5'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:36:54.670268 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-693e7754-eca8-417b-bb8f-b921df19444b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ebd1e625eae34b77bc334aadbe806bf5'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:36:54.678513 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-693e7754-eca8-417b-bb8f-b921df19444b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ebd1e625eae34b77bc334aadbe806bf5'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:36:54.679314 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-693e7754-eca8-417b-bb8f-b921df19444b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ebd1e625eae34b77bc334aadbe806bf5'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:36:54.686558 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:INSERT=2,keystone:DELETE=12,keystone:SELECT=86 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:36:54.703129 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1192/2362] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Nov 2 07:36:54 2022] DELETE /identity/v3/users/ebd1e625eae34b77bc334aadbe806bf5 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:36:55.024165 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99124575-3f9a-4e90-b8ce-0a791d69edfa admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:36:55.026380 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-99124575-3f9a-4e90-b8ce-0a791d69edfa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:36:55.046231 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99124575-3f9a-4e90-b8ce-0a791d69edfa admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:36:55.066733 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:36:55.125689 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cf4d70-fdb8-4cd5-81ca-30cbe4c1fc7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:36:55.126949 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cf4d70-fdb8-4cd5-81ca-30cbe4c1fc7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:55.127225 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cf4d70-fdb8-4cd5-81ca-30cbe4c1fc7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:55.127495 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cf4d70-fdb8-4cd5-81ca-30cbe4c1fc7c admin admin] PATH_INFO: `/v3/projects/1f3a7e48d95a4f638bcef23697b811ee` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:55.128702 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cf4d70-fdb8-4cd5-81ca-30cbe4c1fc7c admin admin] RBAC: Authorizing `identity:delete_project(project_id=1f3a7e48d95a4f638bcef23697b811ee)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:36:55.130634 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cf4d70-fdb8-4cd5-81ca-30cbe4c1fc7c admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:36:55.179444 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-80cf4d70-fdb8-4cd5-81ca-30cbe4c1fc7c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1f3a7e48d95a4f638bcef23697b811ee', 'request_id': 'req-80cf4d70-fdb8-4cd5-81ca-30cbe4c1fc7c'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:36:55.185917 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-80cf4d70-fdb8-4cd5-81ca-30cbe4c1fc7c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1f3a7e48d95a4f638bcef23697b811ee', 'request_id': 'req-80cf4d70-fdb8-4cd5-81ca-30cbe4c1fc7c'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:36:55.194011 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-80cf4d70-fdb8-4cd5-81ca-30cbe4c1fc7c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1f3a7e48d95a4f638bcef23697b811ee', 'request_id': 'req-80cf4d70-fdb8-4cd5-81ca-30cbe4c1fc7c'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:36:55.204185 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-80cf4d70-fdb8-4cd5-81ca-30cbe4c1fc7c admin admin] The token cache is being invalidate because project 1f3a7e48d95a4f638bcef23697b811ee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81501) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:36:55.204461 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-80cf4d70-fdb8-4cd5-81ca-30cbe4c1fc7c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:36:55.205083 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-80cf4d70-fdb8-4cd5-81ca-30cbe4c1fc7c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:36:55.206620 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1171/2363] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Nov 2 07:36:55 2022] DELETE /identity/v3/projects/1f3a7e48d95a4f638bcef23697b811ee => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:36:55.211577 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a81e6a0-6a40-46e7-95a2-b2488a099c62 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:55.211861 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a81e6a0-6a40-46e7-95a2-b2488a099c62 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:55.212128 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a81e6a0-6a40-46e7-95a2-b2488a099c62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:55.231567 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-0a81e6a0-6a40-46e7-95a2-b2488a099c62 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:36:55.232761 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-0a81e6a0-6a40-46e7-95a2-b2488a099c62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:36:55.299968 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1193/2364] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:36:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:36:55.309133 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f740cf0a-e2cf-436a-8bca-a3b0c6a1466f None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:55.309451 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f740cf0a-e2cf-436a-8bca-a3b0c6a1466f None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:55.309728 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f740cf0a-e2cf-436a-8bca-a3b0c6a1466f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:55.330243 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-f740cf0a-e2cf-436a-8bca-a3b0c6a1466f None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:36:55.331470 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-f740cf0a-e2cf-436a-8bca-a3b0c6a1466f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:36:55.339452 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1172/2365] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:36:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:36:55.343808 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a81e6a0-6a40-46e7-95a2-b2488a099c62 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:36:55.354562 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a81e6a0-6a40-46e7-95a2-b2488a099c62 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:36:55.359153 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8719e0c6-65b5-437c-bd6e-679d42eb53b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:36:55.359758 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8719e0c6-65b5-437c-bd6e-679d42eb53b6 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:55.360039 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8719e0c6-65b5-437c-bd6e-679d42eb53b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:55.360342 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8719e0c6-65b5-437c-bd6e-679d42eb53b6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:55.360991 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8719e0c6-65b5-437c-bd6e-679d42eb53b6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:36:55.361346 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8719e0c6-65b5-437c-bd6e-679d42eb53b6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:36:55.362967 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8719e0c6-65b5-437c-bd6e-679d42eb53b6 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:36:55.378215 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1194/2366] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Nov 2 07:36:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:36:55.381616 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f740cf0a-e2cf-436a-8bca-a3b0c6a1466f None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:36:55.387477 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8719e0c6-65b5-437c-bd6e-679d42eb53b6 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:36:55.389287 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-8719e0c6-65b5-437c-bd6e-679d42eb53b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:36:55.392965 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f740cf0a-e2cf-436a-8bca-a3b0c6a1466f None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:36:55.398177 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debe1b22-0d0a-47dc-bf4c-31dd364532cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:36:55.398836 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debe1b22-0d0a-47dc-bf4c-31dd364532cd admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:55.399121 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debe1b22-0d0a-47dc-bf4c-31dd364532cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:55.399395 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debe1b22-0d0a-47dc-bf4c-31dd364532cd admin admin] PATH_INFO: `/v3/projects` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:55.400041 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debe1b22-0d0a-47dc-bf4c-31dd364532cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:36:55.402703 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debe1b22-0d0a-47dc-bf4c-31dd364532cd admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:36:55.435090 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1173/2367] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Nov 2 07:36:55 2022] POST /identity/v3/projects => generated 383 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 07:36:55.439801 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debe1b22-0d0a-47dc-bf4c-31dd364532cd admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:36:55.455841 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8719e0c6-65b5-437c-bd6e-679d42eb53b6 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:36:55.522515 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debe1b22-0d0a-47dc-bf4c-31dd364532cd admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:36:55.524735 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2d9132-3310-47c0-8e4c-82a495543105 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3ec6ecb85f94ef59762877d9ea33ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:36:55.525351 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2d9132-3310-47c0-8e4c-82a495543105 service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:55.525636 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2d9132-3310-47c0-8e4c-82a495543105 service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:55.525912 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2d9132-3310-47c0-8e4c-82a495543105 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:55.527852 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-9a2d9132-3310-47c0-8e4c-82a495543105 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:36:55.528366 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecb0851-e775-47b1-b354-d6327efa5c48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:36:55.529202 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecb0851-e775-47b1-b354-d6327efa5c48 admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:55.529807 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecb0851-e775-47b1-b354-d6327efa5c48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:55.530176 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecb0851-e775-47b1-b354-d6327efa5c48 admin admin] PATH_INFO: `/v3/users` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:55.531035 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecb0851-e775-47b1-b354-d6327efa5c48 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:36:55.534089 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecb0851-e775-47b1-b354-d6327efa5c48 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:36:55.567758 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1174/2368] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Nov 2 07:36:55 2022] POST /identity/v3/users => generated 292 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 07:36:55.568901 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecb0851-e775-47b1-b354-d6327efa5c48 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:36:55.579883 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecb0851-e775-47b1-b354-d6327efa5c48 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:36:55.584954 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:36:55.585711 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8058ab98-f8a5-46a3-b578-0c76e4c702a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:36:55.586115 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8058ab98-f8a5-46a3-b578-0c76e4c702a2 admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:55.586618 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8058ab98-f8a5-46a3-b578-0c76e4c702a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:55.586993 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8058ab98-f8a5-46a3-b578-0c76e4c702a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:55.587762 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8058ab98-f8a5-46a3-b578-0c76e4c702a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:36:55.589424 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8058ab98-f8a5-46a3-b578-0c76e4c702a2 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:36:55.591425 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:36:55.602113 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1175/2369] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:36:55 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:36:55.605398 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2d9132-3310-47c0-8e4c-82a495543105 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:36:55.606935 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8058ab98-f8a5-46a3-b578-0c76e4c702a2 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:36:55.607242 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2d9132-3310-47c0-8e4c-82a495543105 service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:36:55.617756 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8058ab98-f8a5-46a3-b578-0c76e4c702a2 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:36:55.622556 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c52de0-bc65-45b1-acac-2ebbc29c6b55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:36:55.623223 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c52de0-bc65-45b1-acac-2ebbc29c6b55 admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:55.623586 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c52de0-bc65-45b1-acac-2ebbc29c6b55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:55.623911 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c52de0-bc65-45b1-acac-2ebbc29c6b55 admin admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:55.624609 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c52de0-bc65-45b1-acac-2ebbc29c6b55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:36:55.626843 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c52de0-bc65-45b1-acac-2ebbc29c6b55 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:36:55.627305 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1195/2370] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:36:55 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:36:55.638258 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1176/2371] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:36:55 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 07:36:55.643620 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2d9132-3310-47c0-8e4c-82a495543105 service neutron] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:36:55.660676 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2d9132-3310-47c0-8e4c-82a495543105 service neutron] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:36:55.665743 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4390b43c-40d3-4581-8959-1f45ba05e416 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:36:55.666454 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4390b43c-40d3-4581-8959-1f45ba05e416 admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:55.666735 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4390b43c-40d3-4581-8959-1f45ba05e416 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:55.667018 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4390b43c-40d3-4581-8959-1f45ba05e416 admin admin] PATH_INFO: `/v3/projects/8c9fece6c377459ebbe174845d2d9c8d/users/15edb5b3e6bb4ddc9243d82daf4223ac/roles/b494fedd00f24f0b84a529e8672a156a` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:55.683615 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4390b43c-40d3-4581-8959-1f45ba05e416 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c9fece6c377459ebbe174845d2d9c8d, user_id=15edb5b3e6bb4ddc9243d82daf4223ac, role_id=b494fedd00f24f0b84a529e8672a156a)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:36:55.687679 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4390b43c-40d3-4581-8959-1f45ba05e416 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:36:55.701857 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1196/2372] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Nov 2 07:36:55 2022] PUT /identity/v3/projects/8c9fece6c377459ebbe174845d2d9c8d/users/15edb5b3e6bb4ddc9243d82daf4223ac/roles/b494fedd00f24f0b84a529e8672a156a => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 07:36:55.713021 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b18496-35b6-4c72-b3bb-31a01f21972e None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:55.713311 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b18496-35b6-4c72-b3bb-31a01f21972e None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:55.713584 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b18496-35b6-4c72-b3bb-31a01f21972e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:55.741310 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-a1b18496-35b6-4c72-b3bb-31a01f21972e None None] MFA Rules not processed for user `15edb5b3e6bb4ddc9243d82daf4223ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:36:55.742558 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-a1b18496-35b6-4c72-b3bb-31a01f21972e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:36:55.807702 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1177/2373] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:36:55 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:36:55.812193 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4390b43c-40d3-4581-8959-1f45ba05e416 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:36:55.877790 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4390b43c-40d3-4581-8959-1f45ba05e416 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:36:55.882678 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e83420-11ba-4f03-a22d-597615ebc2a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:36:55.883282 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e83420-11ba-4f03-a22d-597615ebc2a8 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:55.883554 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e83420-11ba-4f03-a22d-597615ebc2a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:55.883820 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e83420-11ba-4f03-a22d-597615ebc2a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:55.884449 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e83420-11ba-4f03-a22d-597615ebc2a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:36:55.886325 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e83420-11ba-4f03-a22d-597615ebc2a8 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:36:55.913844 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1197/2374] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Nov 2 07:36:55 2022] POST /identity/v3/projects => generated 385 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 07:36:55.918147 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b18496-35b6-4c72-b3bb-31a01f21972e None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:36:55.929182 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b18496-35b6-4c72-b3bb-31a01f21972e None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:36:55.985939 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1344eb4b-d494-4959-a443-38c2960b52af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:36:55.986500 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1344eb4b-d494-4959-a443-38c2960b52af admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:55.986887 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1344eb4b-d494-4959-a443-38c2960b52af admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:55.987099 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1344eb4b-d494-4959-a443-38c2960b52af admin admin] PATH_INFO: `/v3/users` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:55.987746 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1344eb4b-d494-4959-a443-38c2960b52af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:36:55.989711 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1344eb4b-d494-4959-a443-38c2960b52af admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:36:56.012035 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1178/2375] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Nov 2 07:36:55 2022] POST /identity/v3/users => generated 299 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:36:56.015977 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e83420-11ba-4f03-a22d-597615ebc2a8 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:36:56.026876 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e83420-11ba-4f03-a22d-597615ebc2a8 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:36:56.031441 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6f213d-a520-4dfd-b4e6-290a2e69a0dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:36:56.032120 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6f213d-a520-4dfd-b4e6-290a2e69a0dc admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:56.032880 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6f213d-a520-4dfd-b4e6-290a2e69a0dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:56.033154 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6f213d-a520-4dfd-b4e6-290a2e69a0dc admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:56.033890 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6f213d-a520-4dfd-b4e6-290a2e69a0dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:36:56.035517 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6f213d-a520-4dfd-b4e6-290a2e69a0dc admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:36:56.046670 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1198/2376] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:36:56 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:36:56.051122 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1344eb4b-d494-4959-a443-38c2960b52af admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:36:56.061669 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1344eb4b-d494-4959-a443-38c2960b52af admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:36:56.066754 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f61162b-f50c-406f-9391-08c42b3ed503 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:36:56.067423 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f61162b-f50c-406f-9391-08c42b3ed503 admin admin] REQUEST_METHOD: `PUT` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:56.067773 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f61162b-f50c-406f-9391-08c42b3ed503 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:56.068083 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f61162b-f50c-406f-9391-08c42b3ed503 admin admin] PATH_INFO: `/v3/projects/79f04b0f9e0e47f79abe638582c4e073/users/22e447fbd63d4df098dbc500911c61c3/roles/c4bf019fc6c44c71a6c984f6d0bab75f` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:56.084719 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f61162b-f50c-406f-9391-08c42b3ed503 admin admin] RBAC: Authorizing `identity:create_grant(project_id=79f04b0f9e0e47f79abe638582c4e073, user_id=22e447fbd63d4df098dbc500911c61c3, role_id=c4bf019fc6c44c71a6c984f6d0bab75f)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:36:56.087488 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f61162b-f50c-406f-9391-08c42b3ed503 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:36:56.101473 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1179/2377] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Nov 2 07:36:56 2022] PUT /identity/v3/projects/79f04b0f9e0e47f79abe638582c4e073/users/22e447fbd63d4df098dbc500911c61c3/roles/c4bf019fc6c44c71a6c984f6d0bab75f => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:36:56.105547 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6f213d-a520-4dfd-b4e6-290a2e69a0dc admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:36:56.116868 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6f213d-a520-4dfd-b4e6-290a2e69a0dc admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:36:56.178427 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15212ae8-ff10-416b-95d9-46c072f84b28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:36:56.179049 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15212ae8-ff10-416b-95d9-46c072f84b28 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:56.179328 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15212ae8-ff10-416b-95d9-46c072f84b28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:56.179597 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15212ae8-ff10-416b-95d9-46c072f84b28 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:56.180270 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15212ae8-ff10-416b-95d9-46c072f84b28 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:36:56.181872 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15212ae8-ff10-416b-95d9-46c072f84b28 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:36:56.195689 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1199/2378] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:36:56 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:36:56.196950 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f61162b-f50c-406f-9391-08c42b3ed503 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:36:56.208473 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f61162b-f50c-406f-9391-08c42b3ed503 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:36:56.213353 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fa065f-f434-4fb7-9c2c-365d5df97dbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:36:56.214024 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fa065f-f434-4fb7-9c2c-365d5df97dbf admin admin] REQUEST_METHOD: `PUT` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:56.214305 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fa065f-f434-4fb7-9c2c-365d5df97dbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:56.214690 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fa065f-f434-4fb7-9c2c-365d5df97dbf admin admin] PATH_INFO: `/v3/projects/79f04b0f9e0e47f79abe638582c4e073/users/22e447fbd63d4df098dbc500911c61c3/roles/c4bf019fc6c44c71a6c984f6d0bab75f` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:56.216653 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fa065f-f434-4fb7-9c2c-365d5df97dbf admin admin] RBAC: Authorizing `identity:create_grant(project_id=79f04b0f9e0e47f79abe638582c4e073, user_id=22e447fbd63d4df098dbc500911c61c3, role_id=c4bf019fc6c44c71a6c984f6d0bab75f)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:36:56.219349 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fa065f-f434-4fb7-9c2c-365d5df97dbf admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:36:56.231033 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1180/2379] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Nov 2 07:36:56 2022] PUT /identity/v3/projects/79f04b0f9e0e47f79abe638582c4e073/users/22e447fbd63d4df098dbc500911c61c3/roles/c4bf019fc6c44c71a6c984f6d0bab75f => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:36:56.241877 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1de888-4843-41f5-aa8b-dd222e444d9c None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:56.242061 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1de888-4843-41f5-aa8b-dd222e444d9c None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:56.242338 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1de888-4843-41f5-aa8b-dd222e444d9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:56.273116 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-cd1de888-4843-41f5-aa8b-dd222e444d9c None None] MFA Rules not processed for user `22e447fbd63d4df098dbc500911c61c3`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:36:56.274336 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-cd1de888-4843-41f5-aa8b-dd222e444d9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:36:56.293020 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:36:56.344923 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1200/2380] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:36:56 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:36:56.356392 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fa065f-f434-4fb7-9c2c-365d5df97dbf admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:36:56.358143 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-78fa065f-f434-4fb7-9c2c-365d5df97dbf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:36:56.389463 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:36:56.419576 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fa065f-f434-4fb7-9c2c-365d5df97dbf admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:36:56.424609 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321bf082-7c3e-412b-b6c4-73abff9692cf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43bb326fb82e4d2c83fb2eaf91860dc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:36:56.425217 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321bf082-7c3e-412b-b6c4-73abff9692cf service cinder] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:56.425502 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321bf082-7c3e-412b-b6c4-73abff9692cf service cinder] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:56.425777 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321bf082-7c3e-412b-b6c4-73abff9692cf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:56.486957 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321bf082-7c3e-412b-b6c4-73abff9692cf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:36:56.488399 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321bf082-7c3e-412b-b6c4-73abff9692cf service cinder] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:36:56.497753 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1181/2381] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:36:56 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:37:00.168341 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1de888-4843-41f5-aa8b-dd222e444d9c None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:00.169862 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-cd1de888-4843-41f5-aa8b-dd222e444d9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:37:00.229354 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1de888-4843-41f5-aa8b-dd222e444d9c None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:00.232869 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465f471d-7d1c-4b06-8c87-9faff583d9f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3ec6ecb85f94ef59762877d9ea33ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:00.233461 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465f471d-7d1c-4b06-8c87-9faff583d9f7 service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:00.233735 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465f471d-7d1c-4b06-8c87-9faff583d9f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:00.234001 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465f471d-7d1c-4b06-8c87-9faff583d9f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:00.236460 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-465f471d-7d1c-4b06-8c87-9faff583d9f7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:37:00.299933 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465f471d-7d1c-4b06-8c87-9faff583d9f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:00.301406 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465f471d-7d1c-4b06-8c87-9faff583d9f7 service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:00.311820 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1201/2382] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:37:00 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:37:00.864507 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321bf082-7c3e-412b-b6c4-73abff9692cf service cinder] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:00.874028 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321bf082-7c3e-412b-b6c4-73abff9692cf service cinder] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:00.879032 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732de9f7-e088-4137-b426-acddc8f8f7f2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43bb326fb82e4d2c83fb2eaf91860dc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:00.879750 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732de9f7-e088-4137-b426-acddc8f8f7f2 service cinder] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:00.880029 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732de9f7-e088-4137-b426-acddc8f8f7f2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:00.880299 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732de9f7-e088-4137-b426-acddc8f8f7f2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:00.893283 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732de9f7-e088-4137-b426-acddc8f8f7f2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:00.895610 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732de9f7-e088-4137-b426-acddc8f8f7f2 service cinder] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:00.909561 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1182/2383] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:37:00 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:37:02.017212 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465f471d-7d1c-4b06-8c87-9faff583d9f7 service neutron] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:02.026033 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465f471d-7d1c-4b06-8c87-9faff583d9f7 service neutron] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:02.029703 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afadad0-e1bb-4137-b8ea-3f25cce31346 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3ec6ecb85f94ef59762877d9ea33ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:02.030532 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afadad0-e1bb-4137-b8ea-3f25cce31346 service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:02.030810 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afadad0-e1bb-4137-b8ea-3f25cce31346 service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:02.031076 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afadad0-e1bb-4137-b8ea-3f25cce31346 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:02.032959 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-8afadad0-e1bb-4137-b8ea-3f25cce31346 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:37:02.099628 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afadad0-e1bb-4137-b8ea-3f25cce31346 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:02.101057 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afadad0-e1bb-4137-b8ea-3f25cce31346 service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:02.105977 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:37:02.111267 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1202/2384] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:37:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:37:06.147107 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732de9f7-e088-4137-b426-acddc8f8f7f2 service cinder] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:06.158074 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732de9f7-e088-4137-b426-acddc8f8f7f2 service cinder] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:06.162448 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301aeda2-75f5-43fb-983c-70f09ebfe18f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:06.163118 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301aeda2-75f5-43fb-983c-70f09ebfe18f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:06.163346 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301aeda2-75f5-43fb-983c-70f09ebfe18f admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:06.163649 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301aeda2-75f5-43fb-983c-70f09ebfe18f admin admin] PATH_INFO: `/v3/users/15edb5b3e6bb4ddc9243d82daf4223ac` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:06.164747 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301aeda2-75f5-43fb-983c-70f09ebfe18f admin admin] RBAC: Authorizing `identity:delete_user(user_id=15edb5b3e6bb4ddc9243d82daf4223ac)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:06.166688 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301aeda2-75f5-43fb-983c-70f09ebfe18f admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:06.216141 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-301aeda2-75f5-43fb-983c-70f09ebfe18f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '15edb5b3e6bb4ddc9243d82daf4223ac'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:06.225349 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-301aeda2-75f5-43fb-983c-70f09ebfe18f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '15edb5b3e6bb4ddc9243d82daf4223ac'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:06.226009 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-301aeda2-75f5-43fb-983c-70f09ebfe18f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '15edb5b3e6bb4ddc9243d82daf4223ac'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:06.233468 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-301aeda2-75f5-43fb-983c-70f09ebfe18f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '15edb5b3e6bb4ddc9243d82daf4223ac'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:06.246233 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-301aeda2-75f5-43fb-983c-70f09ebfe18f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '15edb5b3e6bb4ddc9243d82daf4223ac'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:06.252312 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1183/2385] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Nov 2 07:37:06 2022] DELETE /identity/v3/users/15edb5b3e6bb4ddc9243d82daf4223ac => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:37:06.256866 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd667dd-6b5d-4681-8cba-a0b51e2418b8 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:06.257371 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd667dd-6b5d-4681-8cba-a0b51e2418b8 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:06.257583 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd667dd-6b5d-4681-8cba-a0b51e2418b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:06.282706 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-4fd667dd-6b5d-4681-8cba-a0b51e2418b8 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:37:06.284684 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-4fd667dd-6b5d-4681-8cba-a0b51e2418b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:37:06.354404 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1203/2386] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:37:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:37:06.365229 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301aeda2-75f5-43fb-983c-70f09ebfe18f admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:06.366821 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-301aeda2-75f5-43fb-983c-70f09ebfe18f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:37:06.419205 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:37:06.436296 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301aeda2-75f5-43fb-983c-70f09ebfe18f admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:06.440124 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cec2290-5eb7-40a4-9c0c-5aa5a2253fa7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3ec6ecb85f94ef59762877d9ea33ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:06.440727 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cec2290-5eb7-40a4-9c0c-5aa5a2253fa7 service neutron] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:06.441004 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cec2290-5eb7-40a4-9c0c-5aa5a2253fa7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:06.441301 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cec2290-5eb7-40a4-9c0c-5aa5a2253fa7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:06.450291 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cec2290-5eb7-40a4-9c0c-5aa5a2253fa7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:06.451741 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cec2290-5eb7-40a4-9c0c-5aa5a2253fa7 service neutron] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:06.462470 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1184/2387] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:37:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:37:06.795525 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd667dd-6b5d-4681-8cba-a0b51e2418b8 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:06.797497 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-4fd667dd-6b5d-4681-8cba-a0b51e2418b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:37:06.818655 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd667dd-6b5d-4681-8cba-a0b51e2418b8 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:06.823907 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad9e305-0ebf-4874-b2af-f384df3b035e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:06.824745 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad9e305-0ebf-4874-b2af-f384df3b035e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:06.825357 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad9e305-0ebf-4874-b2af-f384df3b035e admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:06.825788 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad9e305-0ebf-4874-b2af-f384df3b035e admin admin] PATH_INFO: `/v3/projects/8c9fece6c377459ebbe174845d2d9c8d` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:06.826959 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad9e305-0ebf-4874-b2af-f384df3b035e admin admin] RBAC: Authorizing `identity:delete_project(project_id=8c9fece6c377459ebbe174845d2d9c8d)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:06.828862 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad9e305-0ebf-4874-b2af-f384df3b035e admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:06.946220 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-9ad9e305-0ebf-4874-b2af-f384df3b035e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8c9fece6c377459ebbe174845d2d9c8d', 'request_id': 'req-9ad9e305-0ebf-4874-b2af-f384df3b035e'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:06.953905 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-9ad9e305-0ebf-4874-b2af-f384df3b035e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8c9fece6c377459ebbe174845d2d9c8d', 'request_id': 'req-9ad9e305-0ebf-4874-b2af-f384df3b035e'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:06.960405 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-9ad9e305-0ebf-4874-b2af-f384df3b035e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8c9fece6c377459ebbe174845d2d9c8d', 'request_id': 'req-9ad9e305-0ebf-4874-b2af-f384df3b035e'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:06.977241 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-9ad9e305-0ebf-4874-b2af-f384df3b035e admin admin] The token cache is being invalidate because project 8c9fece6c377459ebbe174845d2d9c8d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:37:06.977619 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-9ad9e305-0ebf-4874-b2af-f384df3b035e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:06.978337 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-9ad9e305-0ebf-4874-b2af-f384df3b035e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:06.982854 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1204/2388] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Nov 2 07:37:06 2022] DELETE /identity/v3/projects/8c9fece6c377459ebbe174845d2d9c8d => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:37:06.984226 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cec2290-5eb7-40a4-9c0c-5aa5a2253fa7 service neutron] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:06.994602 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cec2290-5eb7-40a4-9c0c-5aa5a2253fa7 service neutron] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:07.086106 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa167a3d-1099-4651-903b-c4dc83e05665 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:07.086822 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa167a3d-1099-4651-903b-c4dc83e05665 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:07.087012 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa167a3d-1099-4651-903b-c4dc83e05665 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:07.087269 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa167a3d-1099-4651-903b-c4dc83e05665 admin admin] PATH_INFO: `/v3/users/22e447fbd63d4df098dbc500911c61c3` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:07.091122 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa167a3d-1099-4651-903b-c4dc83e05665 admin admin] RBAC: Authorizing `identity:delete_user(user_id=22e447fbd63d4df098dbc500911c61c3)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:07.093499 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa167a3d-1099-4651-903b-c4dc83e05665 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:07.173383 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-fa167a3d-1099-4651-903b-c4dc83e05665 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '22e447fbd63d4df098dbc500911c61c3'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:07.181665 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-fa167a3d-1099-4651-903b-c4dc83e05665 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '22e447fbd63d4df098dbc500911c61c3'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:07.182327 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-fa167a3d-1099-4651-903b-c4dc83e05665 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '22e447fbd63d4df098dbc500911c61c3'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:07.190133 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-fa167a3d-1099-4651-903b-c4dc83e05665 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '22e447fbd63d4df098dbc500911c61c3'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:07.207578 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-fa167a3d-1099-4651-903b-c4dc83e05665 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '22e447fbd63d4df098dbc500911c61c3'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:07.216962 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1185/2389] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Nov 2 07:37:06 2022] DELETE /identity/v3/users/22e447fbd63d4df098dbc500911c61c3 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:37:07.506630 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad9e305-0ebf-4874-b2af-f384df3b035e admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:07.508428 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-9ad9e305-0ebf-4874-b2af-f384df3b035e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:37:07.523296 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad9e305-0ebf-4874-b2af-f384df3b035e admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:07.586456 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dab83e7-3419-4230-a899-7c3b9b90e8ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:07.587768 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dab83e7-3419-4230-a899-7c3b9b90e8ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:07.588055 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dab83e7-3419-4230-a899-7c3b9b90e8ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:07.588389 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dab83e7-3419-4230-a899-7c3b9b90e8ca admin admin] PATH_INFO: `/v3/projects/79f04b0f9e0e47f79abe638582c4e073` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:07.589512 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dab83e7-3419-4230-a899-7c3b9b90e8ca admin admin] RBAC: Authorizing `identity:delete_project(project_id=79f04b0f9e0e47f79abe638582c4e073)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:07.591449 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dab83e7-3419-4230-a899-7c3b9b90e8ca admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:07.625456 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:37:07.649821 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-0dab83e7-3419-4230-a899-7c3b9b90e8ca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '79f04b0f9e0e47f79abe638582c4e073', 'request_id': 'req-0dab83e7-3419-4230-a899-7c3b9b90e8ca'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:07.657860 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-0dab83e7-3419-4230-a899-7c3b9b90e8ca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '79f04b0f9e0e47f79abe638582c4e073', 'request_id': 'req-0dab83e7-3419-4230-a899-7c3b9b90e8ca'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:07.664627 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-0dab83e7-3419-4230-a899-7c3b9b90e8ca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '79f04b0f9e0e47f79abe638582c4e073', 'request_id': 'req-0dab83e7-3419-4230-a899-7c3b9b90e8ca'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:07.679204 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-0dab83e7-3419-4230-a899-7c3b9b90e8ca admin admin] The token cache is being invalidate because project 79f04b0f9e0e47f79abe638582c4e073 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:37:07.679495 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-0dab83e7-3419-4230-a899-7c3b9b90e8ca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:07.680138 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-0dab83e7-3419-4230-a899-7c3b9b90e8ca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:07.681820 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1205/2390] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Nov 2 07:37:07 2022] DELETE /identity/v3/projects/79f04b0f9e0e47f79abe638582c4e073 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 07:37:07.687589 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ae35eb-8a10-460a-a258-2e157d331925 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:07.687881 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ae35eb-8a10-460a-a258-2e157d331925 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:07.688153 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ae35eb-8a10-460a-a258-2e157d331925 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:07.708960 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-35ae35eb-8a10-460a-a258-2e157d331925 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:37:07.710081 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-35ae35eb-8a10-460a-a258-2e157d331925 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:37:07.775368 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1186/2391] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:37:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:37:07.779283 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f5de68-d433-4e58-b9d1-31e260d4fdbc None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:07.779568 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f5de68-d433-4e58-b9d1-31e260d4fdbc None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:07.779835 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f5de68-d433-4e58-b9d1-31e260d4fdbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:07.800045 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-14f5de68-d433-4e58-b9d1-31e260d4fdbc None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:37:07.801346 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-14f5de68-d433-4e58-b9d1-31e260d4fdbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:37:07.809347 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1206/2392] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:37:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:37:07.814189 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ae35eb-8a10-460a-a258-2e157d331925 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:07.819106 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:37:07.825502 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ae35eb-8a10-460a-a258-2e157d331925 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:07.831495 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226cda1f-8fe8-423a-a634-e0085f7db4a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:07.832181 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226cda1f-8fe8-423a-a634-e0085f7db4a4 admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:07.832462 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226cda1f-8fe8-423a-a634-e0085f7db4a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:07.832731 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226cda1f-8fe8-423a-a634-e0085f7db4a4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:07.833369 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226cda1f-8fe8-423a-a634-e0085f7db4a4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:37:07.833671 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226cda1f-8fe8-423a-a634-e0085f7db4a4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:07.835296 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226cda1f-8fe8-423a-a634-e0085f7db4a4 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:07.851539 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1187/2393] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Nov 2 07:37:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:37:07.852701 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f5de68-d433-4e58-b9d1-31e260d4fdbc None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:07.861920 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f5de68-d433-4e58-b9d1-31e260d4fdbc None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:07.866828 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102deea3-b688-4b81-aca5-363ff8710db3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:07.867572 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102deea3-b688-4b81-aca5-363ff8710db3 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:07.867849 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102deea3-b688-4b81-aca5-363ff8710db3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:07.868122 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102deea3-b688-4b81-aca5-363ff8710db3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:07.868750 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102deea3-b688-4b81-aca5-363ff8710db3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:07.870660 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102deea3-b688-4b81-aca5-363ff8710db3 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:07.902985 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1207/2394] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Nov 2 07:37:07 2022] POST /identity/v3/projects => generated 371 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:37:07.904172 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226cda1f-8fe8-423a-a634-e0085f7db4a4 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:07.914291 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226cda1f-8fe8-423a-a634-e0085f7db4a4 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:07.971177 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190a89fd-33b9-426c-8ce8-4df06c0d269f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:07.971800 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190a89fd-33b9-426c-8ce8-4df06c0d269f admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:07.972076 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190a89fd-33b9-426c-8ce8-4df06c0d269f admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:07.972345 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190a89fd-33b9-426c-8ce8-4df06c0d269f admin admin] PATH_INFO: `/v3/users` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:07.972960 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190a89fd-33b9-426c-8ce8-4df06c0d269f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:07.974899 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190a89fd-33b9-426c-8ce8-4df06c0d269f admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:07.997408 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1188/2395] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Nov 2 07:37:07 2022] POST /identity/v3/users => generated 286 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:37:08.002398 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102deea3-b688-4b81-aca5-363ff8710db3 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:08.012661 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102deea3-b688-4b81-aca5-363ff8710db3 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:08.017301 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca27957d-24ae-477c-9f97-54b9bf35d274 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:08.017897 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca27957d-24ae-477c-9f97-54b9bf35d274 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:08.018177 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca27957d-24ae-477c-9f97-54b9bf35d274 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:08.018475 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca27957d-24ae-477c-9f97-54b9bf35d274 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:08.019109 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca27957d-24ae-477c-9f97-54b9bf35d274 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:08.020670 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca27957d-24ae-477c-9f97-54b9bf35d274 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:08.031026 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1208/2396] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:37:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:37:08.035522 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190a89fd-33b9-426c-8ce8-4df06c0d269f admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:08.045538 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190a89fd-33b9-426c-8ce8-4df06c0d269f admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:08.050825 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f78fa5c-f774-414a-a5b7-e3b59f8eddc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:08.051416 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f78fa5c-f774-414a-a5b7-e3b59f8eddc7 admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:08.051720 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f78fa5c-f774-414a-a5b7-e3b59f8eddc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:08.051986 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f78fa5c-f774-414a-a5b7-e3b59f8eddc7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:08.052678 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f78fa5c-f774-414a-a5b7-e3b59f8eddc7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:08.054262 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f78fa5c-f774-414a-a5b7-e3b59f8eddc7 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:08.065210 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1189/2397] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:37:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 07:37:08.069486 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca27957d-24ae-477c-9f97-54b9bf35d274 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:08.080245 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca27957d-24ae-477c-9f97-54b9bf35d274 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:08.085199 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3f24a4-23c7-437b-a270-d2b948cb1248 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:08.085979 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3f24a4-23c7-437b-a270-d2b948cb1248 admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:08.086261 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3f24a4-23c7-437b-a270-d2b948cb1248 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:08.086569 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3f24a4-23c7-437b-a270-d2b948cb1248 admin admin] PATH_INFO: `/v3/projects/fe89304f5f2944d9ac627252d8080c70/users/9018a6623b934d3585a8cf2dfcb602f5/roles/b494fedd00f24f0b84a529e8672a156a` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:08.101571 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3f24a4-23c7-437b-a270-d2b948cb1248 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe89304f5f2944d9ac627252d8080c70, user_id=9018a6623b934d3585a8cf2dfcb602f5, role_id=b494fedd00f24f0b84a529e8672a156a)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:08.104236 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3f24a4-23c7-437b-a270-d2b948cb1248 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:08.129801 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1209/2398] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Nov 2 07:37:08 2022] PUT /identity/v3/projects/fe89304f5f2944d9ac627252d8080c70/users/9018a6623b934d3585a8cf2dfcb602f5/roles/b494fedd00f24f0b84a529e8672a156a => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:37:08.142022 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85102d0-f041-4800-9a3e-65162ec0417e None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:08.142333 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85102d0-f041-4800-9a3e-65162ec0417e None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:08.142694 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85102d0-f041-4800-9a3e-65162ec0417e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:08.186740 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-c85102d0-f041-4800-9a3e-65162ec0417e None None] MFA Rules not processed for user `9018a6623b934d3585a8cf2dfcb602f5`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:37:08.188014 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-c85102d0-f041-4800-9a3e-65162ec0417e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:37:08.260006 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1190/2399] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:37:08 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:37:08.272294 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3f24a4-23c7-437b-a270-d2b948cb1248 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:08.274532 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-cb3f24a4-23c7-437b-a270-d2b948cb1248 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:37:08.357141 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3f24a4-23c7-437b-a270-d2b948cb1248 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:08.361931 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d693b0e-8336-4681-bd96-97811ef510a6 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5353f2af0f44d668e2a269647aa1dc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:08.362578 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d693b0e-8336-4681-bd96-97811ef510a6 service glance] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:08.362859 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d693b0e-8336-4681-bd96-97811ef510a6 service glance] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:08.363131 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d693b0e-8336-4681-bd96-97811ef510a6 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:08.371988 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d693b0e-8336-4681-bd96-97811ef510a6 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:08.373455 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d693b0e-8336-4681-bd96-97811ef510a6 service glance] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:08.384296 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1210/2400] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:37:08 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:37:10.532095 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85102d0-f041-4800-9a3e-65162ec0417e None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:10.536258 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-c85102d0-f041-4800-9a3e-65162ec0417e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:37:10.597199 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d693b0e-8336-4681-bd96-97811ef510a6 service glance] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:10.598774 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-8d693b0e-8336-4681-bd96-97811ef510a6 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:37:10.620147 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:37:10.645499 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85102d0-f041-4800-9a3e-65162ec0417e None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:10.653883 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e00046-6c2a-4e5d-9bdc-c64a69303440 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2fc6a156c54d0c9d325901d0f7f6fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:10.654620 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e00046-6c2a-4e5d-9bdc-c64a69303440 service placement] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:10.655219 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e00046-6c2a-4e5d-9bdc-c64a69303440 service placement] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:10.655512 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e00046-6c2a-4e5d-9bdc-c64a69303440 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:10.657814 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-c5e00046-6c2a-4e5d-9bdc-c64a69303440 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:37:10.661802 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:DELETE=6,keystone:SELECT=91,keystone:INSERT=3 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:37:10.677494 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e00046-6c2a-4e5d-9bdc-c64a69303440 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:10.678256 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d693b0e-8336-4681-bd96-97811ef510a6 service glance] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:10.679394 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e00046-6c2a-4e5d-9bdc-c64a69303440 service placement] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:10.691296 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d22314-0eed-414f-b6e0-44cbe940d964 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:10.692062 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d22314-0eed-414f-b6e0-44cbe940d964 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:10.692457 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d22314-0eed-414f-b6e0-44cbe940d964 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:10.692891 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d22314-0eed-414f-b6e0-44cbe940d964 admin admin] PATH_INFO: `/v3/users/040b847b0db549a2a865deb7be6a8da7` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:10.693557 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1191/2401] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:37:10 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 07:37:10.694315 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d22314-0eed-414f-b6e0-44cbe940d964 admin admin] RBAC: Authorizing `identity:delete_user(user_id=040b847b0db549a2a865deb7be6a8da7)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:10.696444 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d22314-0eed-414f-b6e0-44cbe940d964 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:10.751771 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-11d22314-0eed-414f-b6e0-44cbe940d964 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '040b847b0db549a2a865deb7be6a8da7'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:10.759279 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-11d22314-0eed-414f-b6e0-44cbe940d964 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '040b847b0db549a2a865deb7be6a8da7'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:10.759887 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-11d22314-0eed-414f-b6e0-44cbe940d964 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '040b847b0db549a2a865deb7be6a8da7'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:10.767596 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-11d22314-0eed-414f-b6e0-44cbe940d964 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '040b847b0db549a2a865deb7be6a8da7'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:10.768508 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-11d22314-0eed-414f-b6e0-44cbe940d964 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '040b847b0db549a2a865deb7be6a8da7'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:10.785660 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1211/2402] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Nov 2 07:37:10 2022] DELETE /identity/v3/users/040b847b0db549a2a865deb7be6a8da7 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 07:37:10.968261 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e00046-6c2a-4e5d-9bdc-c64a69303440 service placement] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:10.981946 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e00046-6c2a-4e5d-9bdc-c64a69303440 service placement] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:11.046818 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e12251a-cf6b-4969-b058-02188078cf9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:11.047459 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e12251a-cf6b-4969-b058-02188078cf9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:11.048832 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e12251a-cf6b-4969-b058-02188078cf9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:11.049125 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e12251a-cf6b-4969-b058-02188078cf9e admin admin] PATH_INFO: `/v3/projects/82915a36bbdb4b5998fd98833e234ad2` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:11.050332 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e12251a-cf6b-4969-b058-02188078cf9e admin admin] RBAC: Authorizing `identity:delete_project(project_id=82915a36bbdb4b5998fd98833e234ad2)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:11.052310 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e12251a-cf6b-4969-b058-02188078cf9e admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:11.101450 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-5e12251a-cf6b-4969-b058-02188078cf9e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '82915a36bbdb4b5998fd98833e234ad2', 'request_id': 'req-5e12251a-cf6b-4969-b058-02188078cf9e'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:11.107438 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-5e12251a-cf6b-4969-b058-02188078cf9e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '82915a36bbdb4b5998fd98833e234ad2', 'request_id': 'req-5e12251a-cf6b-4969-b058-02188078cf9e'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:11.115852 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-5e12251a-cf6b-4969-b058-02188078cf9e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '82915a36bbdb4b5998fd98833e234ad2', 'request_id': 'req-5e12251a-cf6b-4969-b058-02188078cf9e'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:11.127045 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-5e12251a-cf6b-4969-b058-02188078cf9e admin admin] The token cache is being invalidate because project 82915a36bbdb4b5998fd98833e234ad2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81501) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:37:11.127376 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-5e12251a-cf6b-4969-b058-02188078cf9e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:11.128003 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-5e12251a-cf6b-4969-b058-02188078cf9e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:11.129758 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1192/2403] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Nov 2 07:37:10 2022] DELETE /identity/v3/projects/82915a36bbdb4b5998fd98833e234ad2 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:37:11.134308 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d22314-0eed-414f-b6e0-44cbe940d964 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:11.135734 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-11d22314-0eed-414f-b6e0-44cbe940d964 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:37:11.150626 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d22314-0eed-414f-b6e0-44cbe940d964 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:11.208417 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b2ef18-721d-475d-a60b-69c051ad870d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:11.209045 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b2ef18-721d-475d-a60b-69c051ad870d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:11.209324 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b2ef18-721d-475d-a60b-69c051ad870d admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:11.209615 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b2ef18-721d-475d-a60b-69c051ad870d admin admin] PATH_INFO: `/v3/users/7fd3e52fad0f4ff9af317adaa3e1aba4` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:11.210758 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b2ef18-721d-475d-a60b-69c051ad870d admin admin] RBAC: Authorizing `identity:delete_user(user_id=7fd3e52fad0f4ff9af317adaa3e1aba4)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:11.212652 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b2ef18-721d-475d-a60b-69c051ad870d admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:11.260844 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-e5b2ef18-721d-475d-a60b-69c051ad870d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7fd3e52fad0f4ff9af317adaa3e1aba4'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:11.268227 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-e5b2ef18-721d-475d-a60b-69c051ad870d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7fd3e52fad0f4ff9af317adaa3e1aba4'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:11.268897 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-e5b2ef18-721d-475d-a60b-69c051ad870d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7fd3e52fad0f4ff9af317adaa3e1aba4'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:11.275814 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-e5b2ef18-721d-475d-a60b-69c051ad870d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7fd3e52fad0f4ff9af317adaa3e1aba4'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:11.276432 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-e5b2ef18-721d-475d-a60b-69c051ad870d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7fd3e52fad0f4ff9af317adaa3e1aba4'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:11.284704 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:37:11.297442 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1212/2404] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Nov 2 07:37:11 2022] DELETE /identity/v3/users/7fd3e52fad0f4ff9af317adaa3e1aba4 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 07:37:11.480080 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e12251a-cf6b-4969-b058-02188078cf9e admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:11.481798 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-5e12251a-cf6b-4969-b058-02188078cf9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:37:11.497173 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e12251a-cf6b-4969-b058-02188078cf9e admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:11.556579 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5b679a-6040-4ddb-88f9-094d1e82b983 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:11.557330 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5b679a-6040-4ddb-88f9-094d1e82b983 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:11.557605 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5b679a-6040-4ddb-88f9-094d1e82b983 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:11.557874 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5b679a-6040-4ddb-88f9-094d1e82b983 admin admin] PATH_INFO: `/v3/projects/d69c92f22f8d42bdb324ed5d021b56e3` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:11.559051 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5b679a-6040-4ddb-88f9-094d1e82b983 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d69c92f22f8d42bdb324ed5d021b56e3)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:11.560999 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5b679a-6040-4ddb-88f9-094d1e82b983 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:11.583807 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:37:11.612117 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-ad5b679a-6040-4ddb-88f9-094d1e82b983 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd69c92f22f8d42bdb324ed5d021b56e3', 'request_id': 'req-ad5b679a-6040-4ddb-88f9-094d1e82b983'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:11.618599 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-ad5b679a-6040-4ddb-88f9-094d1e82b983 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd69c92f22f8d42bdb324ed5d021b56e3', 'request_id': 'req-ad5b679a-6040-4ddb-88f9-094d1e82b983'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:11.626027 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-ad5b679a-6040-4ddb-88f9-094d1e82b983 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd69c92f22f8d42bdb324ed5d021b56e3', 'request_id': 'req-ad5b679a-6040-4ddb-88f9-094d1e82b983'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:11.637520 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-ad5b679a-6040-4ddb-88f9-094d1e82b983 admin admin] The token cache is being invalidate because project d69c92f22f8d42bdb324ed5d021b56e3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81501) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:37:11.637811 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-ad5b679a-6040-4ddb-88f9-094d1e82b983 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:11.638472 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-ad5b679a-6040-4ddb-88f9-094d1e82b983 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:11.639975 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1193/2405] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Nov 2 07:37:11 2022] DELETE /identity/v3/projects/d69c92f22f8d42bdb324ed5d021b56e3 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 07:37:21.296702 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=3,keystone:DELETE=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:37:21.634228 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=16,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:37:33.073735 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b2ef18-721d-475d-a60b-69c051ad870d admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:33.075174 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-e5b2ef18-721d-475d-a60b-69c051ad870d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:37:33.216508 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b2ef18-721d-475d-a60b-69c051ad870d admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:33.275984 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961e5a86-78f1-4ef4-bab9-0c4d0bfd2812 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2fc6a156c54d0c9d325901d0f7f6fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:33.276581 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961e5a86-78f1-4ef4-bab9-0c4d0bfd2812 service placement] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:33.276855 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961e5a86-78f1-4ef4-bab9-0c4d0bfd2812 service placement] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:33.277120 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961e5a86-78f1-4ef4-bab9-0c4d0bfd2812 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:33.278866 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-961e5a86-78f1-4ef4-bab9-0c4d0bfd2812 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:37:33.290442 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961e5a86-78f1-4ef4-bab9-0c4d0bfd2812 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:33.291997 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961e5a86-78f1-4ef4-bab9-0c4d0bfd2812 service placement] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:33.301409 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1213/2406] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:37:33 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:37:36.268537 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5b679a-6040-4ddb-88f9-094d1e82b983 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:36.269921 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-ad5b679a-6040-4ddb-88f9-094d1e82b983 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:37:36.386992 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5b679a-6040-4ddb-88f9-094d1e82b983 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:36.391734 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2378c9-ff71-43d3-8931-aa2e075f08db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:36.392362 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2378c9-ff71-43d3-8931-aa2e075f08db admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:36.392640 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2378c9-ff71-43d3-8931-aa2e075f08db admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:36.392911 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2378c9-ff71-43d3-8931-aa2e075f08db admin admin] PATH_INFO: `/v3/users/9018a6623b934d3585a8cf2dfcb602f5` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:36.394024 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2378c9-ff71-43d3-8931-aa2e075f08db admin admin] RBAC: Authorizing `identity:delete_user(user_id=9018a6623b934d3585a8cf2dfcb602f5)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:36.395964 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2378c9-ff71-43d3-8931-aa2e075f08db admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:36.441013 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-aa2378c9-ff71-43d3-8931-aa2e075f08db admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9018a6623b934d3585a8cf2dfcb602f5'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:36.448559 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-aa2378c9-ff71-43d3-8931-aa2e075f08db admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9018a6623b934d3585a8cf2dfcb602f5'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:36.449224 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-aa2378c9-ff71-43d3-8931-aa2e075f08db admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9018a6623b934d3585a8cf2dfcb602f5'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:36.455983 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-aa2378c9-ff71-43d3-8931-aa2e075f08db admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9018a6623b934d3585a8cf2dfcb602f5'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:36.468218 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-aa2378c9-ff71-43d3-8931-aa2e075f08db admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9018a6623b934d3585a8cf2dfcb602f5'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:36.473895 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1194/2407] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Nov 2 07:37:36 2022] DELETE /identity/v3/users/9018a6623b934d3585a8cf2dfcb602f5 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:37:36.479458 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1347802-67b5-40d0-b7b3-2d69702347e6 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:36.479941 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1347802-67b5-40d0-b7b3-2d69702347e6 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:36.480058 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1347802-67b5-40d0-b7b3-2d69702347e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:36.507787 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-b1347802-67b5-40d0-b7b3-2d69702347e6 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:37:36.508964 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-b1347802-67b5-40d0-b7b3-2d69702347e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:37:36.520469 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:37:36.567814 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1214/2408] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:37:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:37:36.578206 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2378c9-ff71-43d3-8931-aa2e075f08db admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:36.579614 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-aa2378c9-ff71-43d3-8931-aa2e075f08db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:37:36.610337 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:37:36.638449 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2378c9-ff71-43d3-8931-aa2e075f08db admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:36.642004 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bed574-6221-4c5c-beb7-eede248e01ff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3ec6ecb85f94ef59762877d9ea33ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:36.642635 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bed574-6221-4c5c-beb7-eede248e01ff service neutron] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:36.642915 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bed574-6221-4c5c-beb7-eede248e01ff service neutron] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:36.643184 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bed574-6221-4c5c-beb7-eede248e01ff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:36.652630 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bed574-6221-4c5c-beb7-eede248e01ff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:36.654050 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bed574-6221-4c5c-beb7-eede248e01ff service neutron] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:36.663564 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1195/2409] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:37:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:37:36.934408 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1347802-67b5-40d0-b7b3-2d69702347e6 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:36.935827 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-b1347802-67b5-40d0-b7b3-2d69702347e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:37:36.950175 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1347802-67b5-40d0-b7b3-2d69702347e6 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:36.956630 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac8805b-0ac8-492a-b4ea-b8bf0337ebc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:36.957822 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac8805b-0ac8-492a-b4ea-b8bf0337ebc9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:36.958263 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac8805b-0ac8-492a-b4ea-b8bf0337ebc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:36.958615 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac8805b-0ac8-492a-b4ea-b8bf0337ebc9 admin admin] PATH_INFO: `/v3/projects/fe89304f5f2944d9ac627252d8080c70` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:36.959747 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac8805b-0ac8-492a-b4ea-b8bf0337ebc9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fe89304f5f2944d9ac627252d8080c70)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:36.961631 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac8805b-0ac8-492a-b4ea-b8bf0337ebc9 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:37.008835 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-eac8805b-0ac8-492a-b4ea-b8bf0337ebc9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fe89304f5f2944d9ac627252d8080c70', 'request_id': 'req-eac8805b-0ac8-492a-b4ea-b8bf0337ebc9'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:37.015239 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-eac8805b-0ac8-492a-b4ea-b8bf0337ebc9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fe89304f5f2944d9ac627252d8080c70', 'request_id': 'req-eac8805b-0ac8-492a-b4ea-b8bf0337ebc9'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:37.020763 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-eac8805b-0ac8-492a-b4ea-b8bf0337ebc9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fe89304f5f2944d9ac627252d8080c70', 'request_id': 'req-eac8805b-0ac8-492a-b4ea-b8bf0337ebc9'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:37.033553 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-eac8805b-0ac8-492a-b4ea-b8bf0337ebc9 admin admin] The token cache is being invalidate because project fe89304f5f2944d9ac627252d8080c70 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:37:37.033840 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-eac8805b-0ac8-492a-b4ea-b8bf0337ebc9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:37.034490 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-eac8805b-0ac8-492a-b4ea-b8bf0337ebc9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:37.035987 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1215/2410] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Nov 2 07:37:36 2022] DELETE /identity/v3/projects/fe89304f5f2944d9ac627252d8080c70 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:37:37.041377 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e916a0d2-3291-4882-9fe2-1ea76d33c3c8 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:37.041663 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e916a0d2-3291-4882-9fe2-1ea76d33c3c8 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:37.041935 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e916a0d2-3291-4882-9fe2-1ea76d33c3c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:37.089867 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-e916a0d2-3291-4882-9fe2-1ea76d33c3c8 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:37:37.091012 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-e916a0d2-3291-4882-9fe2-1ea76d33c3c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:37:37.149311 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1196/2411] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:37:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:37:37.214260 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb56f6b6-154b-4e94-8891-f6237feb4f22 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:37.214578 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb56f6b6-154b-4e94-8891-f6237feb4f22 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:37.214855 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb56f6b6-154b-4e94-8891-f6237feb4f22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:37.235838 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-bb56f6b6-154b-4e94-8891-f6237feb4f22 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:37:37.237054 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-bb56f6b6-154b-4e94-8891-f6237feb4f22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:37:37.245760 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1216/2412] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:37:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:37:37.250241 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e916a0d2-3291-4882-9fe2-1ea76d33c3c8 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:37.260562 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e916a0d2-3291-4882-9fe2-1ea76d33c3c8 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:37.266051 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05c2792-6506-4927-8332-d4b863187a0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:37.266761 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05c2792-6506-4927-8332-d4b863187a0d admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:37.267131 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05c2792-6506-4927-8332-d4b863187a0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:37.267472 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05c2792-6506-4927-8332-d4b863187a0d admin admin] PATH_INFO: `/v3/domains` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:37.268282 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05c2792-6506-4927-8332-d4b863187a0d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:37:37.268657 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05c2792-6506-4927-8332-d4b863187a0d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:37.270315 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05c2792-6506-4927-8332-d4b863187a0d admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:37.282379 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1197/2413] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Nov 2 07:37:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:37:37.287318 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb56f6b6-154b-4e94-8891-f6237feb4f22 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:37.296549 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb56f6b6-154b-4e94-8891-f6237feb4f22 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:37.301168 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309ced5f-ef7d-471a-b717-428a7de6278d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:37.301775 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309ced5f-ef7d-471a-b717-428a7de6278d admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:37.302052 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309ced5f-ef7d-471a-b717-428a7de6278d admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:37.303081 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309ced5f-ef7d-471a-b717-428a7de6278d admin admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:37.303721 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309ced5f-ef7d-471a-b717-428a7de6278d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:37.305657 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309ced5f-ef7d-471a-b717-428a7de6278d admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:37.334470 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1217/2414] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Nov 2 07:37:37 2022] POST /identity/v3/projects => generated 361 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:37:37.338740 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05c2792-6506-4927-8332-d4b863187a0d admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:37.357537 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05c2792-6506-4927-8332-d4b863187a0d admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:37.415801 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3093b4-f378-4c12-acc3-da595b5c5d4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:37.416406 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3093b4-f378-4c12-acc3-da595b5c5d4a admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:37.416689 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3093b4-f378-4c12-acc3-da595b5c5d4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:37.416976 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3093b4-f378-4c12-acc3-da595b5c5d4a admin admin] PATH_INFO: `/v3/users` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:37.417597 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3093b4-f378-4c12-acc3-da595b5c5d4a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:37.419539 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3093b4-f378-4c12-acc3-da595b5c5d4a admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:37.440122 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1198/2415] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Nov 2 07:37:37 2022] POST /identity/v3/users => generated 281 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 07:37:37.444916 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309ced5f-ef7d-471a-b717-428a7de6278d admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:37.455085 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309ced5f-ef7d-471a-b717-428a7de6278d admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:37.459859 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5c3e65-b4c1-423f-bfd3-7479bb5e3350 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:37.460454 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5c3e65-b4c1-423f-bfd3-7479bb5e3350 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:37.460738 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5c3e65-b4c1-423f-bfd3-7479bb5e3350 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:37.461011 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5c3e65-b4c1-423f-bfd3-7479bb5e3350 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:37.461641 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5c3e65-b4c1-423f-bfd3-7479bb5e3350 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:37.463262 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5c3e65-b4c1-423f-bfd3-7479bb5e3350 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:37.473119 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1218/2416] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:37:37 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:37:37.477210 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3093b4-f378-4c12-acc3-da595b5c5d4a admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:37.487446 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3093b4-f378-4c12-acc3-da595b5c5d4a admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:37.492579 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab80f171-5d81-4f12-a2ed-be712e5167fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:37.493191 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab80f171-5d81-4f12-a2ed-be712e5167fb admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:37.493474 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab80f171-5d81-4f12-a2ed-be712e5167fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:37.493748 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab80f171-5d81-4f12-a2ed-be712e5167fb admin admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:37.494404 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab80f171-5d81-4f12-a2ed-be712e5167fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:37.496038 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab80f171-5d81-4f12-a2ed-be712e5167fb admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:37.506957 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1199/2417] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:37:37 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:37:37.511291 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5c3e65-b4c1-423f-bfd3-7479bb5e3350 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:37.520978 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5c3e65-b4c1-423f-bfd3-7479bb5e3350 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:37.526414 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c101f76a-9273-49f5-9b86-3e43daae625d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:37.527090 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c101f76a-9273-49f5-9b86-3e43daae625d admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:37.527405 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c101f76a-9273-49f5-9b86-3e43daae625d admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:37.527683 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c101f76a-9273-49f5-9b86-3e43daae625d admin admin] PATH_INFO: `/v3/projects/8eaa435f170a4f8aa82dfdb594b5375d/users/a256f945df38473b95ff52d4e93f4986/roles/b494fedd00f24f0b84a529e8672a156a` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:37.542321 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c101f76a-9273-49f5-9b86-3e43daae625d admin admin] RBAC: Authorizing `identity:create_grant(project_id=8eaa435f170a4f8aa82dfdb594b5375d, user_id=a256f945df38473b95ff52d4e93f4986, role_id=b494fedd00f24f0b84a529e8672a156a)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:37.544969 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c101f76a-9273-49f5-9b86-3e43daae625d admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:37.558398 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1219/2418] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Nov 2 07:37:37 2022] PUT /identity/v3/projects/8eaa435f170a4f8aa82dfdb594b5375d/users/a256f945df38473b95ff52d4e93f4986/roles/b494fedd00f24f0b84a529e8672a156a => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:37:37.569009 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cbb00a-d199-4c6a-80ea-372222c913da None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:37.569297 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cbb00a-d199-4c6a-80ea-372222c913da None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:37.569571 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cbb00a-d199-4c6a-80ea-372222c913da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:37.589689 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:37:37.597625 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-d0cbb00a-d199-4c6a-80ea-372222c913da None None] MFA Rules not processed for user `a256f945df38473b95ff52d4e93f4986`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:37:37.599460 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-d0cbb00a-d199-4c6a-80ea-372222c913da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:37:37.668652 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1200/2419] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:37:37 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:37:37.680134 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c101f76a-9273-49f5-9b86-3e43daae625d admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:37.681806 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-c101f76a-9273-49f5-9b86-3e43daae625d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:37:37.722757 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:37:37.739637 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c101f76a-9273-49f5-9b86-3e43daae625d admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:37.743328 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9deae0-4bb5-4a79-84b6-436d266325bf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43bb326fb82e4d2c83fb2eaf91860dc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:37.743927 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9deae0-4bb5-4a79-84b6-436d266325bf service cinder] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:37.744202 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9deae0-4bb5-4a79-84b6-436d266325bf service cinder] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:37.744472 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9deae0-4bb5-4a79-84b6-436d266325bf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:37.752539 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9deae0-4bb5-4a79-84b6-436d266325bf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:37.753950 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9deae0-4bb5-4a79-84b6-436d266325bf service cinder] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:37.763416 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1220/2420] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:37:37 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:37:47.665587 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=27 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:37:47.759181 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=9 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:38:12.999374 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cbb00a-d199-4c6a-80ea-372222c913da None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:38:13.060921 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cbb00a-d199-4c6a-80ea-372222c913da None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:38:13.065357 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23eebb61-7f33-4d67-a083-f201dedd7046 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:38:13.065976 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23eebb61-7f33-4d67-a083-f201dedd7046 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:38:13.066254 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23eebb61-7f33-4d67-a083-f201dedd7046 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:38:13.066550 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23eebb61-7f33-4d67-a083-f201dedd7046 admin admin] PATH_INFO: `/v3/users/a256f945df38473b95ff52d4e93f4986` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:38:13.067673 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23eebb61-7f33-4d67-a083-f201dedd7046 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a256f945df38473b95ff52d4e93f4986)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:38:13.069555 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23eebb61-7f33-4d67-a083-f201dedd7046 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:38:13.116576 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-23eebb61-7f33-4d67-a083-f201dedd7046 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a256f945df38473b95ff52d4e93f4986'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:38:13.125413 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-23eebb61-7f33-4d67-a083-f201dedd7046 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a256f945df38473b95ff52d4e93f4986'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:38:13.126214 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-23eebb61-7f33-4d67-a083-f201dedd7046 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a256f945df38473b95ff52d4e93f4986'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:38:13.133702 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-23eebb61-7f33-4d67-a083-f201dedd7046 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a256f945df38473b95ff52d4e93f4986'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:38:13.146325 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-23eebb61-7f33-4d67-a083-f201dedd7046 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a256f945df38473b95ff52d4e93f4986'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:38:13.152780 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1201/2421] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Nov 2 07:38:12 2022] DELETE /identity/v3/users/a256f945df38473b95ff52d4e93f4986 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:38:13.157163 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119bd525-a786-4730-b840-c00e0967787f None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:38:13.157660 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119bd525-a786-4730-b840-c00e0967787f None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:38:13.157766 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119bd525-a786-4730-b840-c00e0967787f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:38:13.176429 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-119bd525-a786-4730-b840-c00e0967787f None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:38:13.177604 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-119bd525-a786-4730-b840-c00e0967787f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:38:13.234274 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1221/2422] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:38:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:38:13.244767 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23eebb61-7f33-4d67-a083-f201dedd7046 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:38:13.246721 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-23eebb61-7f33-4d67-a083-f201dedd7046 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:38:13.323702 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23eebb61-7f33-4d67-a083-f201dedd7046 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:38:13.328131 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1483fae-b215-4391-9e41-36eae1a5a502 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3ec6ecb85f94ef59762877d9ea33ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:38:13.328735 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1483fae-b215-4391-9e41-36eae1a5a502 service neutron] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:38:13.329015 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1483fae-b215-4391-9e41-36eae1a5a502 service neutron] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:38:13.329287 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1483fae-b215-4391-9e41-36eae1a5a502 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:38:13.338125 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1483fae-b215-4391-9e41-36eae1a5a502 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:38:13.339560 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1483fae-b215-4391-9e41-36eae1a5a502 service neutron] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:38:13.348746 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1202/2423] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:38:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:38:13.611629 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119bd525-a786-4730-b840-c00e0967787f None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:38:13.613040 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-119bd525-a786-4730-b840-c00e0967787f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:38:13.625842 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119bd525-a786-4730-b840-c00e0967787f None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:38:13.631248 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76de1ad9-427f-448e-bfcd-c1480ec4b881 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:38:13.631895 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76de1ad9-427f-448e-bfcd-c1480ec4b881 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:38:13.632179 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76de1ad9-427f-448e-bfcd-c1480ec4b881 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:38:13.632453 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76de1ad9-427f-448e-bfcd-c1480ec4b881 admin admin] PATH_INFO: `/v3/projects/8eaa435f170a4f8aa82dfdb594b5375d` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:38:13.633683 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76de1ad9-427f-448e-bfcd-c1480ec4b881 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8eaa435f170a4f8aa82dfdb594b5375d)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:38:13.635523 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76de1ad9-427f-448e-bfcd-c1480ec4b881 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:38:13.696073 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-76de1ad9-427f-448e-bfcd-c1480ec4b881 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8eaa435f170a4f8aa82dfdb594b5375d', 'request_id': 'req-76de1ad9-427f-448e-bfcd-c1480ec4b881'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:38:13.702134 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-76de1ad9-427f-448e-bfcd-c1480ec4b881 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8eaa435f170a4f8aa82dfdb594b5375d', 'request_id': 'req-76de1ad9-427f-448e-bfcd-c1480ec4b881'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:38:13.707617 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-76de1ad9-427f-448e-bfcd-c1480ec4b881 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8eaa435f170a4f8aa82dfdb594b5375d', 'request_id': 'req-76de1ad9-427f-448e-bfcd-c1480ec4b881'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:38:13.719742 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-76de1ad9-427f-448e-bfcd-c1480ec4b881 admin admin] The token cache is being invalidate because project 8eaa435f170a4f8aa82dfdb594b5375d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:38:13.720025 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-76de1ad9-427f-448e-bfcd-c1480ec4b881 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:38:13.720629 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-76de1ad9-427f-448e-bfcd-c1480ec4b881 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:38:13.722089 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1222/2424] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Nov 2 07:38:13 2022] DELETE /identity/v3/projects/8eaa435f170a4f8aa82dfdb594b5375d => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:38:13.726950 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb36559-2477-4f33-a226-520f149b3987 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:38:13.727233 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb36559-2477-4f33-a226-520f149b3987 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:38:13.727504 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb36559-2477-4f33-a226-520f149b3987 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:38:13.738949 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:38:13.753519 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-1bb36559-2477-4f33-a226-520f149b3987 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:38:13.754809 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-1bb36559-2477-4f33-a226-520f149b3987 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:38:13.812595 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1203/2425] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:38:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:38:13.821745 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8eb3f3-6190-4611-a129-d735efc9b597 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:38:13.822121 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8eb3f3-6190-4611-a129-d735efc9b597 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:38:13.822430 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8eb3f3-6190-4611-a129-d735efc9b597 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:38:13.842082 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-1e8eb3f3-6190-4611-a129-d735efc9b597 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:38:13.843306 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-1e8eb3f3-6190-4611-a129-d735efc9b597 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:38:13.851297 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1223/2426] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:38:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:38:13.856103 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb36559-2477-4f33-a226-520f149b3987 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:38:13.865520 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb36559-2477-4f33-a226-520f149b3987 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:38:13.869960 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297d85e8-b988-40b0-ad3c-7516971f93b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:38:13.870580 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297d85e8-b988-40b0-ad3c-7516971f93b2 admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:38:13.870885 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297d85e8-b988-40b0-ad3c-7516971f93b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:38:13.871199 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297d85e8-b988-40b0-ad3c-7516971f93b2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:38:13.871828 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297d85e8-b988-40b0-ad3c-7516971f93b2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:38:13.872120 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297d85e8-b988-40b0-ad3c-7516971f93b2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:38:13.873689 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297d85e8-b988-40b0-ad3c-7516971f93b2 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:38:13.884822 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1204/2427] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Nov 2 07:38:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:38:13.889684 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8eb3f3-6190-4611-a129-d735efc9b597 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:38:13.898648 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8eb3f3-6190-4611-a129-d735efc9b597 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:38:13.903794 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d57e435-a8e0-4404-8f6e-3f0b2f2f985e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:38:13.904377 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d57e435-a8e0-4404-8f6e-3f0b2f2f985e admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:38:13.904655 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d57e435-a8e0-4404-8f6e-3f0b2f2f985e admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:38:13.904919 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d57e435-a8e0-4404-8f6e-3f0b2f2f985e admin admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:38:13.905547 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d57e435-a8e0-4404-8f6e-3f0b2f2f985e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:38:13.907470 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d57e435-a8e0-4404-8f6e-3f0b2f2f985e admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:38:13.936079 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1224/2428] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Nov 2 07:38:13 2022] POST /identity/v3/projects => generated 373 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:38:13.940242 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297d85e8-b988-40b0-ad3c-7516971f93b2 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:38:13.950691 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297d85e8-b988-40b0-ad3c-7516971f93b2 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:38:14.006145 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fae333-2ba3-4715-a95b-bbcdca0d0aef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:38:14.006889 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fae333-2ba3-4715-a95b-bbcdca0d0aef admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:38:14.007031 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fae333-2ba3-4715-a95b-bbcdca0d0aef admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:38:14.007300 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fae333-2ba3-4715-a95b-bbcdca0d0aef admin admin] PATH_INFO: `/v3/users` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:38:14.007925 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fae333-2ba3-4715-a95b-bbcdca0d0aef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:38:14.009842 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fae333-2ba3-4715-a95b-bbcdca0d0aef admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:38:14.032176 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1205/2429] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Nov 2 07:38:13 2022] POST /identity/v3/users => generated 287 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 07:38:14.036547 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d57e435-a8e0-4404-8f6e-3f0b2f2f985e admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:38:14.046762 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d57e435-a8e0-4404-8f6e-3f0b2f2f985e admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:38:14.051589 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb88af4c-9eec-4007-b38a-9edf5e577cf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:38:14.052326 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb88af4c-9eec-4007-b38a-9edf5e577cf9 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:38:14.052606 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb88af4c-9eec-4007-b38a-9edf5e577cf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:38:14.052884 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb88af4c-9eec-4007-b38a-9edf5e577cf9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:38:14.053521 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb88af4c-9eec-4007-b38a-9edf5e577cf9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:38:14.055180 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb88af4c-9eec-4007-b38a-9edf5e577cf9 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:38:14.067060 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1225/2430] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:38:14 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:38:14.070612 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fae333-2ba3-4715-a95b-bbcdca0d0aef admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:38:14.081200 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fae333-2ba3-4715-a95b-bbcdca0d0aef admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:38:14.085797 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85471268-2dd8-4def-a6c8-ca2fb06c35ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:38:14.086450 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85471268-2dd8-4def-a6c8-ca2fb06c35ac admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:38:14.086732 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85471268-2dd8-4def-a6c8-ca2fb06c35ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:38:14.087003 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85471268-2dd8-4def-a6c8-ca2fb06c35ac admin admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:38:14.087661 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85471268-2dd8-4def-a6c8-ca2fb06c35ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:38:14.089235 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85471268-2dd8-4def-a6c8-ca2fb06c35ac admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:38:14.099860 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1206/2431] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:38:14 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:38:14.103928 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb88af4c-9eec-4007-b38a-9edf5e577cf9 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:38:14.113922 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb88af4c-9eec-4007-b38a-9edf5e577cf9 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:38:14.118585 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e2c578-2491-4be5-955a-59e9cef8df4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:38:14.119217 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e2c578-2491-4be5-955a-59e9cef8df4d admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:38:14.119494 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e2c578-2491-4be5-955a-59e9cef8df4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:38:14.119762 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e2c578-2491-4be5-955a-59e9cef8df4d admin admin] PATH_INFO: `/v3/projects/1673bbba242e418fb382b3aad2044f3d/users/4c7a692819ad4dfeb7f0e4f936ae2c32/roles/b494fedd00f24f0b84a529e8672a156a` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:38:14.134640 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e2c578-2491-4be5-955a-59e9cef8df4d admin admin] RBAC: Authorizing `identity:create_grant(project_id=1673bbba242e418fb382b3aad2044f3d, user_id=4c7a692819ad4dfeb7f0e4f936ae2c32, role_id=b494fedd00f24f0b84a529e8672a156a)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:38:14.138325 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e2c578-2491-4be5-955a-59e9cef8df4d admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:38:14.150932 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1226/2432] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Nov 2 07:38:14 2022] PUT /identity/v3/projects/1673bbba242e418fb382b3aad2044f3d/users/4c7a692819ad4dfeb7f0e4f936ae2c32/roles/b494fedd00f24f0b84a529e8672a156a => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 07:38:14.161518 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3c76e5-1fa7-42e7-b883-430a9a4c28ce None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:38:14.161809 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3c76e5-1fa7-42e7-b883-430a9a4c28ce None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:38:14.162079 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3c76e5-1fa7-42e7-b883-430a9a4c28ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:38:14.189257 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-1e3c76e5-1fa7-42e7-b883-430a9a4c28ce None None] MFA Rules not processed for user `4c7a692819ad4dfeb7f0e4f936ae2c32`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:38:14.190491 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-1e3c76e5-1fa7-42e7-b883-430a9a4c28ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:38:14.248361 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:38:14.254031 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1207/2433] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:38:14 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:38:14.265583 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e2c578-2491-4be5-955a-59e9cef8df4d admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:38:14.267332 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-31e2c578-2491-4be5-955a-59e9cef8df4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:38:14.289657 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:38:14.345316 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e2c578-2491-4be5-955a-59e9cef8df4d admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:38:14.348938 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb15c5e3-91db-4f71-b520-6dd6c57ca828 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43bb326fb82e4d2c83fb2eaf91860dc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:38:14.349535 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb15c5e3-91db-4f71-b520-6dd6c57ca828 service cinder] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:38:14.349811 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb15c5e3-91db-4f71-b520-6dd6c57ca828 service cinder] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:38:14.350096 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb15c5e3-91db-4f71-b520-6dd6c57ca828 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:38:14.358518 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb15c5e3-91db-4f71-b520-6dd6c57ca828 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:38:14.359982 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb15c5e3-91db-4f71-b520-6dd6c57ca828 service cinder] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:38:14.368807 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1227/2434] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:38:14 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:38:24.254920 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=1 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:38:24.333583 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3c76e5-1fa7-42e7-b883-430a9a4c28ce None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:38:24.365134 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=26 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:38:24.396638 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3c76e5-1fa7-42e7-b883-430a9a4c28ce None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:38:24.401148 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a0426b-5b89-41fb-bc24-fa4f4ad3141b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:38:24.401796 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a0426b-5b89-41fb-bc24-fa4f4ad3141b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:38:24.402061 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a0426b-5b89-41fb-bc24-fa4f4ad3141b admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:38:24.402335 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a0426b-5b89-41fb-bc24-fa4f4ad3141b admin admin] PATH_INFO: `/v3/users/4c7a692819ad4dfeb7f0e4f936ae2c32` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:38:24.403500 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a0426b-5b89-41fb-bc24-fa4f4ad3141b admin admin] RBAC: Authorizing `identity:delete_user(user_id=4c7a692819ad4dfeb7f0e4f936ae2c32)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:38:24.405400 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a0426b-5b89-41fb-bc24-fa4f4ad3141b admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:38:24.456086 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-e1a0426b-5b89-41fb-bc24-fa4f4ad3141b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4c7a692819ad4dfeb7f0e4f936ae2c32'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:38:24.462880 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-e1a0426b-5b89-41fb-bc24-fa4f4ad3141b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4c7a692819ad4dfeb7f0e4f936ae2c32'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:38:24.463677 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-e1a0426b-5b89-41fb-bc24-fa4f4ad3141b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4c7a692819ad4dfeb7f0e4f936ae2c32'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:38:24.471827 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-e1a0426b-5b89-41fb-bc24-fa4f4ad3141b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4c7a692819ad4dfeb7f0e4f936ae2c32'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:38:24.485616 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-e1a0426b-5b89-41fb-bc24-fa4f4ad3141b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4c7a692819ad4dfeb7f0e4f936ae2c32'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:38:24.491843 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1208/2435] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Nov 2 07:38:24 2022] DELETE /identity/v3/users/4c7a692819ad4dfeb7f0e4f936ae2c32 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:38:24.496194 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16986134-45c1-4fb0-88e4-e09637a7529c None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:38:24.496543 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16986134-45c1-4fb0-88e4-e09637a7529c None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:38:24.496885 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16986134-45c1-4fb0-88e4-e09637a7529c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:38:24.516817 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-16986134-45c1-4fb0-88e4-e09637a7529c None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:38:24.518085 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-16986134-45c1-4fb0-88e4-e09637a7529c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:38:24.577493 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1228/2436] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:38:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:38:24.587302 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a0426b-5b89-41fb-bc24-fa4f4ad3141b admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:38:24.588723 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-e1a0426b-5b89-41fb-bc24-fa4f4ad3141b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:38:24.654718 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a0426b-5b89-41fb-bc24-fa4f4ad3141b admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:38:24.658873 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d671856-323c-4a32-aa4e-be488df21caa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3ec6ecb85f94ef59762877d9ea33ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:38:24.659477 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d671856-323c-4a32-aa4e-be488df21caa service neutron] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:38:24.659753 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d671856-323c-4a32-aa4e-be488df21caa service neutron] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:38:24.660023 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d671856-323c-4a32-aa4e-be488df21caa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:38:24.669331 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d671856-323c-4a32-aa4e-be488df21caa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:38:24.671032 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d671856-323c-4a32-aa4e-be488df21caa service neutron] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:38:24.682473 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1209/2437] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:38:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:38:24.959986 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16986134-45c1-4fb0-88e4-e09637a7529c None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:38:24.961624 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-16986134-45c1-4fb0-88e4-e09637a7529c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:38:24.975021 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16986134-45c1-4fb0-88e4-e09637a7529c None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:38:24.980455 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8754f893-50bf-4831-ad88-6c4d74870a7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:38:24.981134 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8754f893-50bf-4831-ad88-6c4d74870a7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:38:24.981413 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8754f893-50bf-4831-ad88-6c4d74870a7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:38:24.981685 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8754f893-50bf-4831-ad88-6c4d74870a7b admin admin] PATH_INFO: `/v3/projects/1673bbba242e418fb382b3aad2044f3d` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:38:24.982887 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8754f893-50bf-4831-ad88-6c4d74870a7b admin admin] RBAC: Authorizing `identity:delete_project(project_id=1673bbba242e418fb382b3aad2044f3d)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:38:24.984897 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8754f893-50bf-4831-ad88-6c4d74870a7b admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:38:25.044495 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-8754f893-50bf-4831-ad88-6c4d74870a7b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1673bbba242e418fb382b3aad2044f3d', 'request_id': 'req-8754f893-50bf-4831-ad88-6c4d74870a7b'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:38:25.051972 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-8754f893-50bf-4831-ad88-6c4d74870a7b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1673bbba242e418fb382b3aad2044f3d', 'request_id': 'req-8754f893-50bf-4831-ad88-6c4d74870a7b'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:38:25.057816 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-8754f893-50bf-4831-ad88-6c4d74870a7b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1673bbba242e418fb382b3aad2044f3d', 'request_id': 'req-8754f893-50bf-4831-ad88-6c4d74870a7b'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:38:25.072810 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-8754f893-50bf-4831-ad88-6c4d74870a7b admin admin] The token cache is being invalidate because project 1673bbba242e418fb382b3aad2044f3d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:38:25.073104 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-8754f893-50bf-4831-ad88-6c4d74870a7b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:38:25.073749 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-8754f893-50bf-4831-ad88-6c4d74870a7b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:38:25.075590 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1229/2438] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Nov 2 07:38:24 2022] DELETE /identity/v3/projects/1673bbba242e418fb382b3aad2044f3d => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:38:25.082502 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8eb80a-586b-43f6-b60b-6639791a3fe2 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:38:25.083026 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8eb80a-586b-43f6-b60b-6639791a3fe2 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:38:25.086808 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8eb80a-586b-43f6-b60b-6639791a3fe2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:38:25.115509 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-0b8eb80a-586b-43f6-b60b-6639791a3fe2 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:38:25.116732 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-0b8eb80a-586b-43f6-b60b-6639791a3fe2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:38:25.135748 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:38:25.184277 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1210/2439] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:38:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:38:25.193734 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c17f361-4660-42ad-a6e3-7e8686c81e12 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:38:25.194030 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c17f361-4660-42ad-a6e3-7e8686c81e12 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:38:25.194318 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c17f361-4660-42ad-a6e3-7e8686c81e12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:38:25.214608 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-0c17f361-4660-42ad-a6e3-7e8686c81e12 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:38:25.215824 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-0c17f361-4660-42ad-a6e3-7e8686c81e12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:38:25.223969 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1230/2440] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:38:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:38:25.228227 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8eb80a-586b-43f6-b60b-6639791a3fe2 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:38:25.238196 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8eb80a-586b-43f6-b60b-6639791a3fe2 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:38:25.242874 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef710312-180f-4f4d-b78d-d1f9b47297aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:38:25.243545 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef710312-180f-4f4d-b78d-d1f9b47297aa admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:38:25.243909 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef710312-180f-4f4d-b78d-d1f9b47297aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:38:25.244245 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef710312-180f-4f4d-b78d-d1f9b47297aa admin admin] PATH_INFO: `/v3/domains` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:38:25.244944 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef710312-180f-4f4d-b78d-d1f9b47297aa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:38:25.245307 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef710312-180f-4f4d-b78d-d1f9b47297aa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:38:25.246989 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef710312-180f-4f4d-b78d-d1f9b47297aa admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:38:25.262963 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1211/2441] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Nov 2 07:38:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:38:25.264134 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c17f361-4660-42ad-a6e3-7e8686c81e12 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:38:25.274202 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c17f361-4660-42ad-a6e3-7e8686c81e12 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:38:25.278881 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2480b6a5-524c-4d33-89be-be6c58729c1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:38:25.279564 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2480b6a5-524c-4d33-89be-be6c58729c1e admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:38:25.279903 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2480b6a5-524c-4d33-89be-be6c58729c1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:38:25.280236 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2480b6a5-524c-4d33-89be-be6c58729c1e admin admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:38:25.280930 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2480b6a5-524c-4d33-89be-be6c58729c1e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:38:25.283486 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2480b6a5-524c-4d33-89be-be6c58729c1e admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:38:25.314236 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1231/2442] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Nov 2 07:38:25 2022] POST /identity/v3/projects => generated 383 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:38:25.315435 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef710312-180f-4f4d-b78d-d1f9b47297aa admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:38:25.325763 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef710312-180f-4f4d-b78d-d1f9b47297aa admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:38:25.389338 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469fa1d4-e3bd-48a0-8fe3-5cc9c35bcf93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:38:25.389951 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469fa1d4-e3bd-48a0-8fe3-5cc9c35bcf93 admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:38:25.390233 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469fa1d4-e3bd-48a0-8fe3-5cc9c35bcf93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:38:25.390536 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469fa1d4-e3bd-48a0-8fe3-5cc9c35bcf93 admin admin] PATH_INFO: `/v3/users` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:38:25.391747 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469fa1d4-e3bd-48a0-8fe3-5cc9c35bcf93 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:38:25.396697 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469fa1d4-e3bd-48a0-8fe3-5cc9c35bcf93 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:38:25.417592 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1212/2443] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Nov 2 07:38:25 2022] POST /identity/v3/users => generated 292 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 07:38:25.422018 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2480b6a5-524c-4d33-89be-be6c58729c1e admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:38:25.435377 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2480b6a5-524c-4d33-89be-be6c58729c1e admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:38:25.449701 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed29d0d-43e5-4ef2-b744-a44bc06f31d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:38:25.451065 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed29d0d-43e5-4ef2-b744-a44bc06f31d9 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:38:25.451065 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed29d0d-43e5-4ef2-b744-a44bc06f31d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:38:25.451065 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed29d0d-43e5-4ef2-b744-a44bc06f31d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:38:25.451422 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed29d0d-43e5-4ef2-b744-a44bc06f31d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:38:25.453010 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed29d0d-43e5-4ef2-b744-a44bc06f31d9 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:38:25.470407 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1232/2444] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:38:25 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:38:25.472958 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469fa1d4-e3bd-48a0-8fe3-5cc9c35bcf93 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:38:25.484414 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469fa1d4-e3bd-48a0-8fe3-5cc9c35bcf93 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:38:25.490346 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf51ef7-8f0f-4ab3-91cd-015149a79c4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:38:25.491104 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf51ef7-8f0f-4ab3-91cd-015149a79c4c admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:38:25.491391 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf51ef7-8f0f-4ab3-91cd-015149a79c4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:38:25.491667 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf51ef7-8f0f-4ab3-91cd-015149a79c4c admin admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:38:25.492541 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf51ef7-8f0f-4ab3-91cd-015149a79c4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:38:25.493892 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf51ef7-8f0f-4ab3-91cd-015149a79c4c admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:38:25.507831 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1213/2445] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:38:25 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:38:25.512583 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed29d0d-43e5-4ef2-b744-a44bc06f31d9 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:38:25.537607 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed29d0d-43e5-4ef2-b744-a44bc06f31d9 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:38:25.544252 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2b4bcf-e054-43b9-9562-4a181a4122db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:38:25.545302 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2b4bcf-e054-43b9-9562-4a181a4122db admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:38:25.545864 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2b4bcf-e054-43b9-9562-4a181a4122db admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:38:25.546295 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2b4bcf-e054-43b9-9562-4a181a4122db admin admin] PATH_INFO: `/v3/projects/5c30df4815d74ad3a0805a7e0ee259a6/users/8c67a36d80e845818623cd258a3456fa/roles/b494fedd00f24f0b84a529e8672a156a` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:38:25.565880 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2b4bcf-e054-43b9-9562-4a181a4122db admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c30df4815d74ad3a0805a7e0ee259a6, user_id=8c67a36d80e845818623cd258a3456fa, role_id=b494fedd00f24f0b84a529e8672a156a)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:38:25.569367 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2b4bcf-e054-43b9-9562-4a181a4122db admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:38:25.588158 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1233/2446] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Nov 2 07:38:25 2022] PUT /identity/v3/projects/5c30df4815d74ad3a0805a7e0ee259a6/users/8c67a36d80e845818623cd258a3456fa/roles/b494fedd00f24f0b84a529e8672a156a => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:38:25.593187 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3fac33-30a9-4078-9b41-9726e66a0f76 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:38:25.593475 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3fac33-30a9-4078-9b41-9726e66a0f76 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:38:25.593893 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3fac33-30a9-4078-9b41-9726e66a0f76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:38:25.615315 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-4b3fac33-30a9-4078-9b41-9726e66a0f76 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:38:25.616534 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-4b3fac33-30a9-4078-9b41-9726e66a0f76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:38:25.692916 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1214/2447] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:38:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:38:25.703252 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2b4bcf-e054-43b9-9562-4a181a4122db admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:38:25.705725 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-8c2b4bcf-e054-43b9-9562-4a181a4122db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:38:25.729884 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:38:25.778831 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2b4bcf-e054-43b9-9562-4a181a4122db admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:38:25.783078 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f709836-77e8-415c-81df-48391ec4fade service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3ec6ecb85f94ef59762877d9ea33ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:38:25.783678 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f709836-77e8-415c-81df-48391ec4fade service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:38:25.784057 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f709836-77e8-415c-81df-48391ec4fade service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:38:25.784209 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f709836-77e8-415c-81df-48391ec4fade service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:38:25.793580 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f709836-77e8-415c-81df-48391ec4fade service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:38:25.795024 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f709836-77e8-415c-81df-48391ec4fade service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:38:25.804507 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1234/2448] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:38:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 07:38:26.432951 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b365ba6-50e9-493b-86cb-7f8ae4cc5058 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:38:26.432951 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b365ba6-50e9-493b-86cb-7f8ae4cc5058 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:38:26.432951 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b365ba6-50e9-493b-86cb-7f8ae4cc5058 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:38:26.468064 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-1b365ba6-50e9-493b-86cb-7f8ae4cc5058 None None] MFA Rules not processed for user `8c67a36d80e845818623cd258a3456fa`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:38:26.470913 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-1b365ba6-50e9-493b-86cb-7f8ae4cc5058 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:38:26.489215 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:38:26.540743 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1215/2449] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:38:26 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:38:26.552308 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f709836-77e8-415c-81df-48391ec4fade service neutron] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:38:26.553662 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-3f709836-77e8-415c-81df-48391ec4fade service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:38:26.611489 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f709836-77e8-415c-81df-48391ec4fade service neutron] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:38:26.615087 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2906d31-3be2-4bd6-98b9-6b3c707ea779 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43bb326fb82e4d2c83fb2eaf91860dc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:38:26.615701 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2906d31-3be2-4bd6-98b9-6b3c707ea779 service cinder] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:38:26.615981 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2906d31-3be2-4bd6-98b9-6b3c707ea779 service cinder] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:38:26.616253 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2906d31-3be2-4bd6-98b9-6b3c707ea779 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:38:26.624783 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2906d31-3be2-4bd6-98b9-6b3c707ea779 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:38:26.626716 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2906d31-3be2-4bd6-98b9-6b3c707ea779 service cinder] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:38:26.635599 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1235/2450] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:38:26 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:38:28.013041 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec031eae-6d63-4f29-92e4-f9e8f30dbf9d None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:38:28.013572 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec031eae-6d63-4f29-92e4-f9e8f30dbf9d None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:38:28.013572 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec031eae-6d63-4f29-92e4-f9e8f30dbf9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:38:28.034430 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-ec031eae-6d63-4f29-92e4-f9e8f30dbf9d None None] MFA Rules not processed for user `8c67a36d80e845818623cd258a3456fa`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:38:28.035726 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-ec031eae-6d63-4f29-92e4-f9e8f30dbf9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:38:28.043542 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1216/2451] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:38:28 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:38:32.403377 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2906d31-3be2-4bd6-98b9-6b3c707ea779 service cinder] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:38:32.404773 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-d2906d31-3be2-4bd6-98b9-6b3c707ea779 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:38:32.416728 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2906d31-3be2-4bd6-98b9-6b3c707ea779 service cinder] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:38:32.420243 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d389a535-6030-48bd-80b7-471476286d82 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3ec6ecb85f94ef59762877d9ea33ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:38:32.420829 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d389a535-6030-48bd-80b7-471476286d82 service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:38:32.421107 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d389a535-6030-48bd-80b7-471476286d82 service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:38:32.421377 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d389a535-6030-48bd-80b7-471476286d82 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:38:32.423696 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-d389a535-6030-48bd-80b7-471476286d82 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:38:32.435455 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d389a535-6030-48bd-80b7-471476286d82 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:38:32.436858 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d389a535-6030-48bd-80b7-471476286d82 service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:38:32.446693 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1236/2452] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:38:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:38:38.027940 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=22 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:38:41.756729 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec031eae-6d63-4f29-92e4-f9e8f30dbf9d None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:38:41.758889 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-ec031eae-6d63-4f29-92e4-f9e8f30dbf9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:38:41.778888 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec031eae-6d63-4f29-92e4-f9e8f30dbf9d None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:38:41.784281 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ee4e6a-9f3f-4cd1-8fea-c126ccb68508 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:38:41.784920 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ee4e6a-9f3f-4cd1-8fea-c126ccb68508 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:38:41.785200 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ee4e6a-9f3f-4cd1-8fea-c126ccb68508 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:38:41.785474 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ee4e6a-9f3f-4cd1-8fea-c126ccb68508 admin admin] PATH_INFO: `/v3/users/fe26aea6bb9d4dcebf046da719b2c244` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:38:41.786801 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ee4e6a-9f3f-4cd1-8fea-c126ccb68508 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fe26aea6bb9d4dcebf046da719b2c244)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:38:41.788833 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ee4e6a-9f3f-4cd1-8fea-c126ccb68508 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:38:41.846160 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-02ee4e6a-9f3f-4cd1-8fea-c126ccb68508 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fe26aea6bb9d4dcebf046da719b2c244'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:38:41.855801 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-02ee4e6a-9f3f-4cd1-8fea-c126ccb68508 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fe26aea6bb9d4dcebf046da719b2c244'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:38:41.856821 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-02ee4e6a-9f3f-4cd1-8fea-c126ccb68508 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fe26aea6bb9d4dcebf046da719b2c244'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:38:41.872515 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-02ee4e6a-9f3f-4cd1-8fea-c126ccb68508 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fe26aea6bb9d4dcebf046da719b2c244'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:38:41.890060 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-02ee4e6a-9f3f-4cd1-8fea-c126ccb68508 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fe26aea6bb9d4dcebf046da719b2c244'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:38:41.921941 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1217/2453] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Nov 2 07:38:41 2022] DELETE /identity/v3/users/fe26aea6bb9d4dcebf046da719b2c244 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:38:42.190493 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d389a535-6030-48bd-80b7-471476286d82 service neutron] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:38:42.202004 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d389a535-6030-48bd-80b7-471476286d82 service neutron] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:38:42.318689 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e5077d-9865-40fe-9d31-4eb4c4730a2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:38:42.319449 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e5077d-9865-40fe-9d31-4eb4c4730a2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:38:42.319914 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e5077d-9865-40fe-9d31-4eb4c4730a2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:38:42.320504 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e5077d-9865-40fe-9d31-4eb4c4730a2e admin admin] PATH_INFO: `/v3/projects/566514b413e24aaebe69c11ce2159ac9` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:38:42.321758 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e5077d-9865-40fe-9d31-4eb4c4730a2e admin admin] RBAC: Authorizing `identity:delete_project(project_id=566514b413e24aaebe69c11ce2159ac9)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:38:42.323660 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e5077d-9865-40fe-9d31-4eb4c4730a2e admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:38:42.384236 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-27e5077d-9865-40fe-9d31-4eb4c4730a2e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '566514b413e24aaebe69c11ce2159ac9', 'request_id': 'req-27e5077d-9865-40fe-9d31-4eb4c4730a2e'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:38:42.392384 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-27e5077d-9865-40fe-9d31-4eb4c4730a2e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '566514b413e24aaebe69c11ce2159ac9', 'request_id': 'req-27e5077d-9865-40fe-9d31-4eb4c4730a2e'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:38:42.393764 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:38:42.400174 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-27e5077d-9865-40fe-9d31-4eb4c4730a2e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '566514b413e24aaebe69c11ce2159ac9', 'request_id': 'req-27e5077d-9865-40fe-9d31-4eb4c4730a2e'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:38:42.414913 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-27e5077d-9865-40fe-9d31-4eb4c4730a2e admin admin] The token cache is being invalidate because project 566514b413e24aaebe69c11ce2159ac9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:38:42.415201 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-27e5077d-9865-40fe-9d31-4eb4c4730a2e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:38:42.415832 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-27e5077d-9865-40fe-9d31-4eb4c4730a2e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:38:42.417321 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1237/2454] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Nov 2 07:38:42 2022] DELETE /identity/v3/projects/566514b413e24aaebe69c11ce2159ac9 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:38:42.421573 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ee4e6a-9f3f-4cd1-8fea-c126ccb68508 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:38:42.423366 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-02ee4e6a-9f3f-4cd1-8fea-c126ccb68508 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:38:42.436758 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ee4e6a-9f3f-4cd1-8fea-c126ccb68508 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:38:42.494571 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf0e2af-ae84-457a-98da-4e88178579f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:38:42.495205 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf0e2af-ae84-457a-98da-4e88178579f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:38:42.495483 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf0e2af-ae84-457a-98da-4e88178579f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:38:42.495783 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf0e2af-ae84-457a-98da-4e88178579f2 admin admin] PATH_INFO: `/v3/users/905e2a47c1e34de394b4b380521d364e` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:38:42.496897 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf0e2af-ae84-457a-98da-4e88178579f2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=905e2a47c1e34de394b4b380521d364e)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:38:42.498884 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf0e2af-ae84-457a-98da-4e88178579f2 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:38:42.546861 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-3bf0e2af-ae84-457a-98da-4e88178579f2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '905e2a47c1e34de394b4b380521d364e'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:38:42.554535 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-3bf0e2af-ae84-457a-98da-4e88178579f2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '905e2a47c1e34de394b4b380521d364e'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:38:42.555334 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-3bf0e2af-ae84-457a-98da-4e88178579f2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '905e2a47c1e34de394b4b380521d364e'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:38:42.562591 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-3bf0e2af-ae84-457a-98da-4e88178579f2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '905e2a47c1e34de394b4b380521d364e'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:38:42.571225 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:38:42.577112 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-3bf0e2af-ae84-457a-98da-4e88178579f2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '905e2a47c1e34de394b4b380521d364e'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:38:42.586040 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1218/2455] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Nov 2 07:38:42 2022] DELETE /identity/v3/users/905e2a47c1e34de394b4b380521d364e => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 07:38:42.782108 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e5077d-9865-40fe-9d31-4eb4c4730a2e admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:38:42.787433 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-27e5077d-9865-40fe-9d31-4eb4c4730a2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:38:42.803434 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e5077d-9865-40fe-9d31-4eb4c4730a2e admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:38:42.853359 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91d2371-c7a5-466c-b104-846fd6ba00c0 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:38:42.853835 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91d2371-c7a5-466c-b104-846fd6ba00c0 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:38:42.854042 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91d2371-c7a5-466c-b104-846fd6ba00c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:38:42.875381 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae9323c-c542-4374-9116-7fdfbba1ba08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:38:42.880761 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae9323c-c542-4374-9116-7fdfbba1ba08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:38:42.880761 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae9323c-c542-4374-9116-7fdfbba1ba08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:38:42.880761 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae9323c-c542-4374-9116-7fdfbba1ba08 admin admin] PATH_INFO: `/v3/projects/905677f4e8c14a66b988145f6913fca6` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:38:42.882039 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae9323c-c542-4374-9116-7fdfbba1ba08 admin admin] RBAC: Authorizing `identity:delete_project(project_id=905677f4e8c14a66b988145f6913fca6)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:38:42.884016 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae9323c-c542-4374-9116-7fdfbba1ba08 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:38:42.887725 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-a91d2371-c7a5-466c-b104-846fd6ba00c0 None None] MFA Rules not processed for user `ff4736f3d50746b0bf5d6a917b2ca0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:38:42.888967 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-a91d2371-c7a5-466c-b104-846fd6ba00c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:38:42.937362 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-1ae9323c-c542-4374-9116-7fdfbba1ba08 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '905677f4e8c14a66b988145f6913fca6', 'request_id': 'req-1ae9323c-c542-4374-9116-7fdfbba1ba08'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:38:42.959068 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-1ae9323c-c542-4374-9116-7fdfbba1ba08 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '905677f4e8c14a66b988145f6913fca6', 'request_id': 'req-1ae9323c-c542-4374-9116-7fdfbba1ba08'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:38:42.966086 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-1ae9323c-c542-4374-9116-7fdfbba1ba08 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '905677f4e8c14a66b988145f6913fca6', 'request_id': 'req-1ae9323c-c542-4374-9116-7fdfbba1ba08'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:38:42.972092 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1219/2456] 10.222.0.18 () {58 vars in 967 bytes} [Wed Nov 2 07:38:42 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:38:42.980962 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-1ae9323c-c542-4374-9116-7fdfbba1ba08 admin admin] The token cache is being invalidate because project 905677f4e8c14a66b988145f6913fca6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:38:42.980962 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-1ae9323c-c542-4374-9116-7fdfbba1ba08 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:38:42.981676 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-1ae9323c-c542-4374-9116-7fdfbba1ba08 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:38:42.983766 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1238/2457] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Nov 2 07:38:42 2022] DELETE /identity/v3/projects/905677f4e8c14a66b988145f6913fca6 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:38:43.024582 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91d2371-c7a5-466c-b104-846fd6ba00c0 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:38:43.026023 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-a91d2371-c7a5-466c-b104-846fd6ba00c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:38:43.116879 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91d2371-c7a5-466c-b104-846fd6ba00c0 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:38:43.200019 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e04ce0-626a-4b60-9f13-1b731b31ea72 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9116f47889f4db39e0203d6cdae1593', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:38:43.200821 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e04ce0-626a-4b60-9f13-1b731b31ea72 service swift] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:38:43.201185 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e04ce0-626a-4b60-9f13-1b731b31ea72 service swift] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:38:43.201569 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e04ce0-626a-4b60-9f13-1b731b31ea72 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:38:43.208530 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-71e04ce0-626a-4b60-9f13-1b731b31ea72 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:38:43.236860 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:38:43.271451 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e04ce0-626a-4b60-9f13-1b731b31ea72 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:38:43.273094 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e04ce0-626a-4b60-9f13-1b731b31ea72 service swift] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:38:43.283065 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1220/2458] 10.222.0.18 () {60 vars in 1384 bytes} [Wed Nov 2 07:38:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 259 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Nov 02 07:38:50.177045 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae9323c-c542-4374-9116-7fdfbba1ba08 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:38:50.178540 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-1ae9323c-c542-4374-9116-7fdfbba1ba08 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:38:50.245068 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae9323c-c542-4374-9116-7fdfbba1ba08 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:38:50.249662 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54f8cc9-273c-4beb-9189-82858a2f3046 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:38:50.250305 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54f8cc9-273c-4beb-9189-82858a2f3046 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:38:50.251141 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54f8cc9-273c-4beb-9189-82858a2f3046 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:38:50.251475 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54f8cc9-273c-4beb-9189-82858a2f3046 admin admin] PATH_INFO: `/v3/users/b0d0493e92b746eabdd4cb0912fa250c` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:38:50.252544 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54f8cc9-273c-4beb-9189-82858a2f3046 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b0d0493e92b746eabdd4cb0912fa250c)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:38:50.254504 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54f8cc9-273c-4beb-9189-82858a2f3046 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:38:50.269413 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:DELETE=11,keystone:SELECT=87,keystone:INSERT=2 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:38:50.301677 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-b54f8cc9-273c-4beb-9189-82858a2f3046 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b0d0493e92b746eabdd4cb0912fa250c'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:38:50.308862 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-b54f8cc9-273c-4beb-9189-82858a2f3046 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b0d0493e92b746eabdd4cb0912fa250c'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:38:50.309458 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-b54f8cc9-273c-4beb-9189-82858a2f3046 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b0d0493e92b746eabdd4cb0912fa250c'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:38:50.317122 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-b54f8cc9-273c-4beb-9189-82858a2f3046 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b0d0493e92b746eabdd4cb0912fa250c'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:38:50.317751 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-b54f8cc9-273c-4beb-9189-82858a2f3046 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b0d0493e92b746eabdd4cb0912fa250c'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:38:50.334798 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1239/2459] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Nov 2 07:38:50 2022] DELETE /identity/v3/users/b0d0493e92b746eabdd4cb0912fa250c => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:38:50.517369 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e04ce0-626a-4b60-9f13-1b731b31ea72 service swift] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:38:50.527926 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e04ce0-626a-4b60-9f13-1b731b31ea72 service swift] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:38:50.590720 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1953e129-f889-4e5d-a927-82f4641e7030 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:38:50.591353 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1953e129-f889-4e5d-a927-82f4641e7030 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:38:50.591653 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1953e129-f889-4e5d-a927-82f4641e7030 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:38:50.591926 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1953e129-f889-4e5d-a927-82f4641e7030 admin admin] PATH_INFO: `/v3/projects/280ce4c6b9174073902820e2e922bedf` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:38:50.593061 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1953e129-f889-4e5d-a927-82f4641e7030 admin admin] RBAC: Authorizing `identity:delete_project(project_id=280ce4c6b9174073902820e2e922bedf)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:38:50.595041 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1953e129-f889-4e5d-a927-82f4641e7030 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:38:50.648433 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-1953e129-f889-4e5d-a927-82f4641e7030 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '280ce4c6b9174073902820e2e922bedf', 'request_id': 'req-1953e129-f889-4e5d-a927-82f4641e7030'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:38:50.655007 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-1953e129-f889-4e5d-a927-82f4641e7030 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '280ce4c6b9174073902820e2e922bedf', 'request_id': 'req-1953e129-f889-4e5d-a927-82f4641e7030'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:38:50.662984 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-1953e129-f889-4e5d-a927-82f4641e7030 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '280ce4c6b9174073902820e2e922bedf', 'request_id': 'req-1953e129-f889-4e5d-a927-82f4641e7030'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:38:50.673970 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-1953e129-f889-4e5d-a927-82f4641e7030 admin admin] The token cache is being invalidate because project 280ce4c6b9174073902820e2e922bedf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81501) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:38:50.674251 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-1953e129-f889-4e5d-a927-82f4641e7030 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:38:50.674902 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-1953e129-f889-4e5d-a927-82f4641e7030 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:38:50.676389 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1221/2460] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Nov 2 07:38:50 2022] DELETE /identity/v3/projects/280ce4c6b9174073902820e2e922bedf => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:38:50.680759 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54f8cc9-273c-4beb-9189-82858a2f3046 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:38:50.682306 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-b54f8cc9-273c-4beb-9189-82858a2f3046 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:38:50.696076 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54f8cc9-273c-4beb-9189-82858a2f3046 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:38:50.755832 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cec41b1-097b-4cd1-b5b4-031e2ed61852 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:38:50.756590 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cec41b1-097b-4cd1-b5b4-031e2ed61852 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:38:50.756869 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cec41b1-097b-4cd1-b5b4-031e2ed61852 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:38:50.757278 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cec41b1-097b-4cd1-b5b4-031e2ed61852 admin admin] PATH_INFO: `/v3/users/4735f72a2f1c4dbdb6b244aadeb71d00` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:38:50.758639 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cec41b1-097b-4cd1-b5b4-031e2ed61852 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4735f72a2f1c4dbdb6b244aadeb71d00)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:38:50.761051 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cec41b1-097b-4cd1-b5b4-031e2ed61852 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:38:50.814434 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-9cec41b1-097b-4cd1-b5b4-031e2ed61852 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4735f72a2f1c4dbdb6b244aadeb71d00'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:38:50.824090 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-9cec41b1-097b-4cd1-b5b4-031e2ed61852 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4735f72a2f1c4dbdb6b244aadeb71d00'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:38:50.824871 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-9cec41b1-097b-4cd1-b5b4-031e2ed61852 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4735f72a2f1c4dbdb6b244aadeb71d00'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:38:50.834595 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-9cec41b1-097b-4cd1-b5b4-031e2ed61852 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4735f72a2f1c4dbdb6b244aadeb71d00'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:38:50.835328 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-9cec41b1-097b-4cd1-b5b4-031e2ed61852 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4735f72a2f1c4dbdb6b244aadeb71d00'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:38:50.854739 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1240/2461] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Nov 2 07:38:50 2022] DELETE /identity/v3/users/4735f72a2f1c4dbdb6b244aadeb71d00 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 07:38:51.056829 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1953e129-f889-4e5d-a927-82f4641e7030 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:38:51.059038 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-1953e129-f889-4e5d-a927-82f4641e7030 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:38:51.075531 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1953e129-f889-4e5d-a927-82f4641e7030 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:38:51.152980 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:38:51.161877 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e90fc2-440b-474d-8b9f-6da4681e9270 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:38:51.163291 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e90fc2-440b-474d-8b9f-6da4681e9270 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:38:51.163669 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e90fc2-440b-474d-8b9f-6da4681e9270 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:38:51.164005 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e90fc2-440b-474d-8b9f-6da4681e9270 admin admin] PATH_INFO: `/v3/projects/546abb669f364409a3d449dd8056a2c6` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:38:51.165566 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e90fc2-440b-474d-8b9f-6da4681e9270 admin admin] RBAC: Authorizing `identity:delete_project(project_id=546abb669f364409a3d449dd8056a2c6)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:38:51.168038 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e90fc2-440b-474d-8b9f-6da4681e9270 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:38:51.233250 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-19e90fc2-440b-474d-8b9f-6da4681e9270 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '546abb669f364409a3d449dd8056a2c6', 'request_id': 'req-19e90fc2-440b-474d-8b9f-6da4681e9270'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:38:51.239744 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-19e90fc2-440b-474d-8b9f-6da4681e9270 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '546abb669f364409a3d449dd8056a2c6', 'request_id': 'req-19e90fc2-440b-474d-8b9f-6da4681e9270'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:38:51.247227 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-19e90fc2-440b-474d-8b9f-6da4681e9270 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '546abb669f364409a3d449dd8056a2c6', 'request_id': 'req-19e90fc2-440b-474d-8b9f-6da4681e9270'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:38:51.258119 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-19e90fc2-440b-474d-8b9f-6da4681e9270 admin admin] The token cache is being invalidate because project 546abb669f364409a3d449dd8056a2c6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81501) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:38:51.258439 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-19e90fc2-440b-474d-8b9f-6da4681e9270 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:38:51.259091 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-19e90fc2-440b-474d-8b9f-6da4681e9270 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:38:51.260576 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1222/2462] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Nov 2 07:38:51 2022] DELETE /identity/v3/projects/546abb669f364409a3d449dd8056a2c6 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:39:00.851301 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=74,keystone:DELETE=19,keystone:INSERT=2 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:39:01.254783 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=26,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:39:19.200067 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cec41b1-097b-4cd1-b5b4-031e2ed61852 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:39:19.201459 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-9cec41b1-097b-4cd1-b5b4-031e2ed61852 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:39:19.259785 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cec41b1-097b-4cd1-b5b4-031e2ed61852 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:39:19.308135 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a832cc-95fa-453d-92ff-b792788a50f8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43bb326fb82e4d2c83fb2eaf91860dc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:39:19.308734 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a832cc-95fa-453d-92ff-b792788a50f8 service cinder] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:39:19.309015 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a832cc-95fa-453d-92ff-b792788a50f8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:39:19.309290 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a832cc-95fa-453d-92ff-b792788a50f8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:39:19.311011 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-b9a832cc-95fa-453d-92ff-b792788a50f8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:39:19.393338 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a832cc-95fa-453d-92ff-b792788a50f8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:39:19.394797 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a832cc-95fa-453d-92ff-b792788a50f8 service cinder] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:39:19.415426 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1241/2463] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:39:19 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:39:22.008273 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e90fc2-440b-474d-8b9f-6da4681e9270 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:39:22.009766 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-19e90fc2-440b-474d-8b9f-6da4681e9270 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:39:22.068591 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e90fc2-440b-474d-8b9f-6da4681e9270 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:39:22.072189 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61176f0a-3c8d-40e4-9570-22dfc1128966 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3ec6ecb85f94ef59762877d9ea33ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:39:22.072788 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61176f0a-3c8d-40e4-9570-22dfc1128966 service neutron] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:39:22.073069 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61176f0a-3c8d-40e4-9570-22dfc1128966 service neutron] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:39:22.073343 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61176f0a-3c8d-40e4-9570-22dfc1128966 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:39:22.075059 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-61176f0a-3c8d-40e4-9570-22dfc1128966 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:39:22.089280 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61176f0a-3c8d-40e4-9570-22dfc1128966 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:39:22.090773 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61176f0a-3c8d-40e4-9570-22dfc1128966 service neutron] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:39:22.102070 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1223/2464] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:39:22 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:39:29.412881 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=83 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:39:32.096670 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=26 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:39:32.121252 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ef64de-3329-4af5-84f1-d0daf37ff65e None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:39:32.121720 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ef64de-3329-4af5-84f1-d0daf37ff65e None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:39:32.121844 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ef64de-3329-4af5-84f1-d0daf37ff65e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:39:32.140633 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-e7ef64de-3329-4af5-84f1-d0daf37ff65e None None] MFA Rules not processed for user `8c67a36d80e845818623cd258a3456fa`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:39:32.141804 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-e7ef64de-3329-4af5-84f1-d0daf37ff65e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:39:32.197253 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1242/2465] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:39:32 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:39:42.190559 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=25 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:39:47.540616 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accc5054-1ec6-4ac7-922c-571ba46b41ae None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:39:47.541130 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accc5054-1ec6-4ac7-922c-571ba46b41ae None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:39:47.541250 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accc5054-1ec6-4ac7-922c-571ba46b41ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:39:47.560805 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-accc5054-1ec6-4ac7-922c-571ba46b41ae None None] MFA Rules not processed for user `33c63dc278ac4eb79d2a4553fcbeda4f`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:39:47.561989 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-accc5054-1ec6-4ac7-922c-571ba46b41ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:39:47.614805 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1224/2466] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:39:47 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:39:50.075961 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ef64de-3329-4af5-84f1-d0daf37ff65e None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:39:50.087253 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ef64de-3329-4af5-84f1-d0daf37ff65e None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:39:50.090860 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855b93bf-cea9-49ae-a3b3-4a1d8752ab9a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3ec6ecb85f94ef59762877d9ea33ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:39:50.091475 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855b93bf-cea9-49ae-a3b3-4a1d8752ab9a service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:39:50.091754 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855b93bf-cea9-49ae-a3b3-4a1d8752ab9a service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:39:50.092023 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855b93bf-cea9-49ae-a3b3-4a1d8752ab9a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:39:50.093819 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-855b93bf-cea9-49ae-a3b3-4a1d8752ab9a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:39:50.104977 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855b93bf-cea9-49ae-a3b3-4a1d8752ab9a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:39:50.106533 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855b93bf-cea9-49ae-a3b3-4a1d8752ab9a service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:39:50.115144 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1243/2467] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:39:50 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:39:57.608218 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=25 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:40:00.111157 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=6 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:40:00.870390 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accc5054-1ec6-4ac7-922c-571ba46b41ae None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:40:00.871916 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-accc5054-1ec6-4ac7-922c-571ba46b41ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:40:00.888225 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accc5054-1ec6-4ac7-922c-571ba46b41ae None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:40:00.894013 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda9d926-c184-4d00-9f3c-983720ac4ea9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e48cba3f804cf08b2477f01ae95179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:40:00.894679 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda9d926-c184-4d00-9f3c-983720ac4ea9 service nova] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:40:00.894962 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda9d926-c184-4d00-9f3c-983720ac4ea9 service nova] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:40:00.895455 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda9d926-c184-4d00-9f3c-983720ac4ea9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:40:00.897920 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-eda9d926-c184-4d00-9f3c-983720ac4ea9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:40:00.910452 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda9d926-c184-4d00-9f3c-983720ac4ea9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:40:00.912730 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda9d926-c184-4d00-9f3c-983720ac4ea9 service nova] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:40:00.923753 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1225/2468] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:40:00 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:40:03.547827 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855b93bf-cea9-49ae-a3b3-4a1d8752ab9a service neutron] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:40:03.562979 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855b93bf-cea9-49ae-a3b3-4a1d8752ab9a service neutron] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:40:03.569152 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356c15b3-1d02-46e3-9f49-ffcd0599da3d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e48cba3f804cf08b2477f01ae95179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:40:03.569857 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356c15b3-1d02-46e3-9f49-ffcd0599da3d service nova] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:40:03.570210 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356c15b3-1d02-46e3-9f49-ffcd0599da3d service nova] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:40:03.570595 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356c15b3-1d02-46e3-9f49-ffcd0599da3d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:40:03.572423 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-356c15b3-1d02-46e3-9f49-ffcd0599da3d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:40:03.585513 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356c15b3-1d02-46e3-9f49-ffcd0599da3d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:40:03.587209 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356c15b3-1d02-46e3-9f49-ffcd0599da3d service nova] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:40:03.597964 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1244/2469] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:40:03 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:40:10.919300 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=6 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:40:13.594032 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=6 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:40:30.982902 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda9d926-c184-4d00-9f3c-983720ac4ea9 service nova] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:40:30.984432 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-eda9d926-c184-4d00-9f3c-983720ac4ea9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:40:31.048393 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda9d926-c184-4d00-9f3c-983720ac4ea9 service nova] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:40:31.052847 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef660075-3ae4-4198-bd6e-9fb4842a1f60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:40:31.053474 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef660075-3ae4-4198-bd6e-9fb4842a1f60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:40:31.053757 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef660075-3ae4-4198-bd6e-9fb4842a1f60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:40:31.054031 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef660075-3ae4-4198-bd6e-9fb4842a1f60 admin admin] PATH_INFO: `/v3/users/8c67a36d80e845818623cd258a3456fa` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:40:31.055167 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef660075-3ae4-4198-bd6e-9fb4842a1f60 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8c67a36d80e845818623cd258a3456fa)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:40:31.058041 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef660075-3ae4-4198-bd6e-9fb4842a1f60 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:40:31.103506 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-ef660075-3ae4-4198-bd6e-9fb4842a1f60 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8c67a36d80e845818623cd258a3456fa'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:40:31.110352 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-ef660075-3ae4-4198-bd6e-9fb4842a1f60 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8c67a36d80e845818623cd258a3456fa'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:40:31.110974 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-ef660075-3ae4-4198-bd6e-9fb4842a1f60 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8c67a36d80e845818623cd258a3456fa'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:40:31.117683 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-ef660075-3ae4-4198-bd6e-9fb4842a1f60 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8c67a36d80e845818623cd258a3456fa'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:40:31.130093 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-ef660075-3ae4-4198-bd6e-9fb4842a1f60 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8c67a36d80e845818623cd258a3456fa'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:40:31.136123 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1226/2470] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Nov 2 07:40:30 2022] DELETE /identity/v3/users/8c67a36d80e845818623cd258a3456fa => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:40:31.310440 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356c15b3-1d02-46e3-9f49-ffcd0599da3d service nova] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:40:31.322134 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356c15b3-1d02-46e3-9f49-ffcd0599da3d service nova] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:40:31.377980 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4a2e63-fb80-4527-babb-ff0e2b54fb79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:40:31.378637 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4a2e63-fb80-4527-babb-ff0e2b54fb79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:40:31.378916 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4a2e63-fb80-4527-babb-ff0e2b54fb79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:40:31.379185 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4a2e63-fb80-4527-babb-ff0e2b54fb79 admin admin] PATH_INFO: `/v3/projects/5c30df4815d74ad3a0805a7e0ee259a6` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:40:31.380272 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4a2e63-fb80-4527-babb-ff0e2b54fb79 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5c30df4815d74ad3a0805a7e0ee259a6)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:40:31.382137 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4a2e63-fb80-4527-babb-ff0e2b54fb79 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:40:31.427837 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-1e4a2e63-fb80-4527-babb-ff0e2b54fb79 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5c30df4815d74ad3a0805a7e0ee259a6', 'request_id': 'req-1e4a2e63-fb80-4527-babb-ff0e2b54fb79'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:40:31.433717 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-1e4a2e63-fb80-4527-babb-ff0e2b54fb79 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5c30df4815d74ad3a0805a7e0ee259a6', 'request_id': 'req-1e4a2e63-fb80-4527-babb-ff0e2b54fb79'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:40:31.439071 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-1e4a2e63-fb80-4527-babb-ff0e2b54fb79 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5c30df4815d74ad3a0805a7e0ee259a6', 'request_id': 'req-1e4a2e63-fb80-4527-babb-ff0e2b54fb79'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:40:31.450738 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-1e4a2e63-fb80-4527-babb-ff0e2b54fb79 admin admin] The token cache is being invalidate because project 5c30df4815d74ad3a0805a7e0ee259a6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:40:31.451014 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-1e4a2e63-fb80-4527-babb-ff0e2b54fb79 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:40:31.451618 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-1e4a2e63-fb80-4527-babb-ff0e2b54fb79 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:40:31.453069 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1245/2471] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Nov 2 07:40:31 2022] DELETE /identity/v3/projects/5c30df4815d74ad3a0805a7e0ee259a6 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:40:41.132380 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=53,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:40:41.447838 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=51,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:41:03.995678 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef660075-3ae4-4198-bd6e-9fb4842a1f60 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:41:03.998001 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-ef660075-3ae4-4198-bd6e-9fb4842a1f60 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:41:04.093001 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef660075-3ae4-4198-bd6e-9fb4842a1f60 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:41:04.180399 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc8ddc3-1e94-44fe-ac4d-bba5daff1c0f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e48cba3f804cf08b2477f01ae95179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:41:04.181025 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc8ddc3-1e94-44fe-ac4d-bba5daff1c0f service nova] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:41:04.181308 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc8ddc3-1e94-44fe-ac4d-bba5daff1c0f service nova] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:41:04.181582 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc8ddc3-1e94-44fe-ac4d-bba5daff1c0f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:41:04.184301 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-cbc8ddc3-1e94-44fe-ac4d-bba5daff1c0f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:41:04.201525 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc8ddc3-1e94-44fe-ac4d-bba5daff1c0f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:41:04.203346 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc8ddc3-1e94-44fe-ac4d-bba5daff1c0f service nova] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:41:04.219162 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1227/2472] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:41:03 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:41:12.162251 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4a2e63-fb80-4527-babb-ff0e2b54fb79 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:41:12.163687 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-1e4a2e63-fb80-4527-babb-ff0e2b54fb79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:41:12.231751 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4a2e63-fb80-4527-babb-ff0e2b54fb79 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:41:12.236485 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ca8c27-799b-4114-b0af-b2d2d0df31e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:41:12.237113 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ca8c27-799b-4114-b0af-b2d2d0df31e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:41:12.237395 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ca8c27-799b-4114-b0af-b2d2d0df31e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:41:12.237667 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ca8c27-799b-4114-b0af-b2d2d0df31e9 admin admin] PATH_INFO: `/v3/users/33c63dc278ac4eb79d2a4553fcbeda4f` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:41:12.238851 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ca8c27-799b-4114-b0af-b2d2d0df31e9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=33c63dc278ac4eb79d2a4553fcbeda4f)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:41:12.240803 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ca8c27-799b-4114-b0af-b2d2d0df31e9 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:41:12.287326 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-77ca8c27-799b-4114-b0af-b2d2d0df31e9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '33c63dc278ac4eb79d2a4553fcbeda4f'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:41:12.294927 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-77ca8c27-799b-4114-b0af-b2d2d0df31e9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '33c63dc278ac4eb79d2a4553fcbeda4f'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:41:12.295516 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-77ca8c27-799b-4114-b0af-b2d2d0df31e9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '33c63dc278ac4eb79d2a4553fcbeda4f'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:41:12.302590 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-77ca8c27-799b-4114-b0af-b2d2d0df31e9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '33c63dc278ac4eb79d2a4553fcbeda4f'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:41:12.303215 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-77ca8c27-799b-4114-b0af-b2d2d0df31e9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '33c63dc278ac4eb79d2a4553fcbeda4f'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:41:12.320881 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1246/2473] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Nov 2 07:41:12 2022] DELETE /identity/v3/users/33c63dc278ac4eb79d2a4553fcbeda4f => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:41:12.497659 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc8ddc3-1e94-44fe-ac4d-bba5daff1c0f service nova] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:41:12.510341 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc8ddc3-1e94-44fe-ac4d-bba5daff1c0f service nova] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:41:12.568261 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f49f276-c171-485a-97fb-32d53a280872 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:41:12.568887 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f49f276-c171-485a-97fb-32d53a280872 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:41:12.569173 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f49f276-c171-485a-97fb-32d53a280872 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:41:12.569452 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f49f276-c171-485a-97fb-32d53a280872 admin admin] PATH_INFO: `/v3/projects/86cca0dcb21a47e483458b36e33f5036` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:41:12.570616 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f49f276-c171-485a-97fb-32d53a280872 admin admin] RBAC: Authorizing `identity:delete_project(project_id=86cca0dcb21a47e483458b36e33f5036)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:41:12.572508 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f49f276-c171-485a-97fb-32d53a280872 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:41:12.603178 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:41:12.620742 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-5f49f276-c171-485a-97fb-32d53a280872 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '86cca0dcb21a47e483458b36e33f5036', 'request_id': 'req-5f49f276-c171-485a-97fb-32d53a280872'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:41:12.626757 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-5f49f276-c171-485a-97fb-32d53a280872 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '86cca0dcb21a47e483458b36e33f5036', 'request_id': 'req-5f49f276-c171-485a-97fb-32d53a280872'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:41:12.634034 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-5f49f276-c171-485a-97fb-32d53a280872 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '86cca0dcb21a47e483458b36e33f5036', 'request_id': 'req-5f49f276-c171-485a-97fb-32d53a280872'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:41:12.646859 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-5f49f276-c171-485a-97fb-32d53a280872 admin admin] The token cache is being invalidate because project 86cca0dcb21a47e483458b36e33f5036 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81501) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:41:12.646989 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-5f49f276-c171-485a-97fb-32d53a280872 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:41:12.647608 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-5f49f276-c171-485a-97fb-32d53a280872 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:41:12.649089 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1228/2474] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Nov 2 07:41:12 2022] DELETE /identity/v3/projects/86cca0dcb21a47e483458b36e33f5036 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:41:12.654607 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c9bdcb-d37c-4f3f-8633-3b90350704dd None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:41:12.654892 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c9bdcb-d37c-4f3f-8633-3b90350704dd None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:41:12.655161 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c9bdcb-d37c-4f3f-8633-3b90350704dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:41:12.674901 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-b8c9bdcb-d37c-4f3f-8633-3b90350704dd None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:41:12.676085 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-b8c9bdcb-d37c-4f3f-8633-3b90350704dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:41:12.735185 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1247/2475] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:41:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:41:12.742935 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5108361-2204-40d3-85b5-b2e1fa9733c2 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:41:12.743220 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5108361-2204-40d3-85b5-b2e1fa9733c2 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:41:12.743493 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5108361-2204-40d3-85b5-b2e1fa9733c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:41:12.762897 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-a5108361-2204-40d3-85b5-b2e1fa9733c2 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:41:12.764351 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-a5108361-2204-40d3-85b5-b2e1fa9733c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:41:12.772118 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1229/2476] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:41:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:41:12.776220 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c9bdcb-d37c-4f3f-8633-3b90350704dd None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:41:12.786203 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c9bdcb-d37c-4f3f-8633-3b90350704dd None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:41:12.790891 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca391a92-5067-444c-9b59-5f51bc1869e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:41:12.791478 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca391a92-5067-444c-9b59-5f51bc1869e6 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:41:12.791754 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca391a92-5067-444c-9b59-5f51bc1869e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:41:12.792019 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca391a92-5067-444c-9b59-5f51bc1869e6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:41:12.792651 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca391a92-5067-444c-9b59-5f51bc1869e6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:41:12.793562 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca391a92-5067-444c-9b59-5f51bc1869e6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:41:12.795147 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca391a92-5067-444c-9b59-5f51bc1869e6 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:41:12.806350 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1248/2477] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Nov 2 07:41:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:41:12.810703 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5108361-2204-40d3-85b5-b2e1fa9733c2 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:41:12.819720 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5108361-2204-40d3-85b5-b2e1fa9733c2 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:41:12.824954 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2ccaf0-dff9-4cb7-986f-a28fcc1d1857 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:41:12.825530 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2ccaf0-dff9-4cb7-986f-a28fcc1d1857 admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:41:12.825807 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2ccaf0-dff9-4cb7-986f-a28fcc1d1857 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:41:12.826066 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2ccaf0-dff9-4cb7-986f-a28fcc1d1857 admin admin] PATH_INFO: `/v3/projects` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:41:12.826729 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2ccaf0-dff9-4cb7-986f-a28fcc1d1857 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:41:12.828596 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2ccaf0-dff9-4cb7-986f-a28fcc1d1857 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:41:12.864508 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1230/2478] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Nov 2 07:41:12 2022] POST /identity/v3/projects => generated 373 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:41:12.868728 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca391a92-5067-444c-9b59-5f51bc1869e6 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:41:12.878925 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca391a92-5067-444c-9b59-5f51bc1869e6 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:41:12.882548 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:41:12.937771 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c418f666-f56c-41f9-a2df-d96efe52ea3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:41:12.938432 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c418f666-f56c-41f9-a2df-d96efe52ea3a admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:41:12.938719 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c418f666-f56c-41f9-a2df-d96efe52ea3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:41:12.938995 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c418f666-f56c-41f9-a2df-d96efe52ea3a admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:41:12.939630 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c418f666-f56c-41f9-a2df-d96efe52ea3a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:41:12.941534 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c418f666-f56c-41f9-a2df-d96efe52ea3a admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:41:12.962690 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1249/2479] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Nov 2 07:41:12 2022] POST /identity/v3/users => generated 287 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:41:12.967174 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2ccaf0-dff9-4cb7-986f-a28fcc1d1857 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:41:12.977265 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2ccaf0-dff9-4cb7-986f-a28fcc1d1857 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:41:12.983150 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5399e670-45a4-416a-97a6-1d89a61242ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:41:12.983743 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5399e670-45a4-416a-97a6-1d89a61242ab admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:41:12.984022 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5399e670-45a4-416a-97a6-1d89a61242ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:41:12.984294 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5399e670-45a4-416a-97a6-1d89a61242ab admin admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:41:12.985021 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5399e670-45a4-416a-97a6-1d89a61242ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:41:12.986551 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5399e670-45a4-416a-97a6-1d89a61242ab admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:41:12.997403 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1231/2480] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:41:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:41:13.001548 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c418f666-f56c-41f9-a2df-d96efe52ea3a admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:41:13.011706 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c418f666-f56c-41f9-a2df-d96efe52ea3a admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:41:13.016611 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00423c1a-0252-49bd-bacd-ac2d791f706b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:41:13.017207 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00423c1a-0252-49bd-bacd-ac2d791f706b admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:41:13.017488 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00423c1a-0252-49bd-bacd-ac2d791f706b admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:41:13.017759 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00423c1a-0252-49bd-bacd-ac2d791f706b admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:41:13.018450 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00423c1a-0252-49bd-bacd-ac2d791f706b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:41:13.020055 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00423c1a-0252-49bd-bacd-ac2d791f706b admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:41:13.030843 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1250/2481] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:41:13 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:41:13.035618 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5399e670-45a4-416a-97a6-1d89a61242ab admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:41:13.046593 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5399e670-45a4-416a-97a6-1d89a61242ab admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:41:13.051348 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b33d53-ebc1-4477-b613-d81279b7ce0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:41:13.052026 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b33d53-ebc1-4477-b613-d81279b7ce0f admin admin] REQUEST_METHOD: `PUT` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:41:13.052308 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b33d53-ebc1-4477-b613-d81279b7ce0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:41:13.052581 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b33d53-ebc1-4477-b613-d81279b7ce0f admin admin] PATH_INFO: `/v3/projects/ed115ccc00604c3cb1fb3767d5dd3cec/users/7d552d4a32ce45b2be32332103082b49/roles/b494fedd00f24f0b84a529e8672a156a` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:41:13.068118 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b33d53-ebc1-4477-b613-d81279b7ce0f admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed115ccc00604c3cb1fb3767d5dd3cec, user_id=7d552d4a32ce45b2be32332103082b49, role_id=b494fedd00f24f0b84a529e8672a156a)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:41:13.070807 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b33d53-ebc1-4477-b613-d81279b7ce0f admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:41:13.085403 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1232/2482] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Nov 2 07:41:13 2022] PUT /identity/v3/projects/ed115ccc00604c3cb1fb3767d5dd3cec/users/7d552d4a32ce45b2be32332103082b49/roles/b494fedd00f24f0b84a529e8672a156a => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:41:13.096859 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47c8da3-d413-4cbf-b0a6-2e907cbb3eb2 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:41:13.097149 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47c8da3-d413-4cbf-b0a6-2e907cbb3eb2 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:41:13.097428 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47c8da3-d413-4cbf-b0a6-2e907cbb3eb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:41:13.128013 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-a47c8da3-d413-4cbf-b0a6-2e907cbb3eb2 None None] MFA Rules not processed for user `7d552d4a32ce45b2be32332103082b49`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:41:13.129303 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-a47c8da3-d413-4cbf-b0a6-2e907cbb3eb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:41:13.192452 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1251/2483] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:41:13 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:41:13.203972 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b33d53-ebc1-4477-b613-d81279b7ce0f admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:41:13.205653 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-b9b33d53-ebc1-4477-b613-d81279b7ce0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:41:13.280946 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b33d53-ebc1-4477-b613-d81279b7ce0f admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:41:13.286422 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d28d08-6ae8-4efc-9abc-b8248b51fcdd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e48cba3f804cf08b2477f01ae95179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:41:13.287061 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d28d08-6ae8-4efc-9abc-b8248b51fcdd service nova] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:41:13.287336 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d28d08-6ae8-4efc-9abc-b8248b51fcdd service nova] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:41:13.287605 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d28d08-6ae8-4efc-9abc-b8248b51fcdd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:41:13.295642 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d28d08-6ae8-4efc-9abc-b8248b51fcdd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:41:13.297207 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d28d08-6ae8-4efc-9abc-b8248b51fcdd service nova] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:41:13.305934 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1233/2484] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:41:13 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:41:13.509400 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47c8da3-d413-4cbf-b0a6-2e907cbb3eb2 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:41:13.571248 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47c8da3-d413-4cbf-b0a6-2e907cbb3eb2 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:41:13.576368 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e40eac-502c-4621-8a88-5e5745d6aee4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:41:13.576991 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e40eac-502c-4621-8a88-5e5745d6aee4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:41:13.577272 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e40eac-502c-4621-8a88-5e5745d6aee4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:41:13.577544 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e40eac-502c-4621-8a88-5e5745d6aee4 admin admin] PATH_INFO: `/v3/users/7d552d4a32ce45b2be32332103082b49` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:41:13.578685 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e40eac-502c-4621-8a88-5e5745d6aee4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7d552d4a32ce45b2be32332103082b49)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:41:13.580593 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e40eac-502c-4621-8a88-5e5745d6aee4 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:41:13.597339 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:41:13.628585 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-b5e40eac-502c-4621-8a88-5e5745d6aee4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7d552d4a32ce45b2be32332103082b49'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:41:13.635279 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-b5e40eac-502c-4621-8a88-5e5745d6aee4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7d552d4a32ce45b2be32332103082b49'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:41:13.636257 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-b5e40eac-502c-4621-8a88-5e5745d6aee4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7d552d4a32ce45b2be32332103082b49'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:41:13.644439 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-b5e40eac-502c-4621-8a88-5e5745d6aee4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7d552d4a32ce45b2be32332103082b49'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:41:13.645060 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-b5e40eac-502c-4621-8a88-5e5745d6aee4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7d552d4a32ce45b2be32332103082b49'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:41:13.663203 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1252/2485] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Nov 2 07:41:13 2022] DELETE /identity/v3/users/7d552d4a32ce45b2be32332103082b49 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:41:13.667619 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dda6448-4065-4c8e-9e01-4e689bcbd6b7 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:41:13.668187 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dda6448-4065-4c8e-9e01-4e689bcbd6b7 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:41:13.668187 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dda6448-4065-4c8e-9e01-4e689bcbd6b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:41:13.686715 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-9dda6448-4065-4c8e-9e01-4e689bcbd6b7 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:41:13.687882 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-9dda6448-4065-4c8e-9e01-4e689bcbd6b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:41:13.707291 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:DELETE=6,keystone:SELECT=91,keystone:INSERT=3 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:41:13.748622 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1234/2486] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:41:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:41:13.758404 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e40eac-502c-4621-8a88-5e5745d6aee4 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:41:13.759776 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-b5e40eac-502c-4621-8a88-5e5745d6aee4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:41:13.815684 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e40eac-502c-4621-8a88-5e5745d6aee4 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:41:13.819347 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4fa5c9-e9b8-456c-8165-b90e9e83f742 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3ec6ecb85f94ef59762877d9ea33ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:41:13.819970 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4fa5c9-e9b8-456c-8165-b90e9e83f742 service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:41:13.820254 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4fa5c9-e9b8-456c-8165-b90e9e83f742 service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:41:13.820525 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4fa5c9-e9b8-456c-8165-b90e9e83f742 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:41:13.829988 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4fa5c9-e9b8-456c-8165-b90e9e83f742 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:41:13.831452 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4fa5c9-e9b8-456c-8165-b90e9e83f742 service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:41:13.841325 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1253/2487] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:41:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:41:14.106228 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dda6448-4065-4c8e-9e01-4e689bcbd6b7 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:41:14.107695 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-9dda6448-4065-4c8e-9e01-4e689bcbd6b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:41:14.120635 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dda6448-4065-4c8e-9e01-4e689bcbd6b7 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:41:14.125767 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4f9c93-cbd6-47c1-a0fb-bc79e75d1372 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:41:14.126430 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4f9c93-cbd6-47c1-a0fb-bc79e75d1372 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:41:14.126710 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4f9c93-cbd6-47c1-a0fb-bc79e75d1372 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:41:14.126978 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4f9c93-cbd6-47c1-a0fb-bc79e75d1372 admin admin] PATH_INFO: `/v3/projects/ed115ccc00604c3cb1fb3767d5dd3cec` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:41:14.128672 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4f9c93-cbd6-47c1-a0fb-bc79e75d1372 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ed115ccc00604c3cb1fb3767d5dd3cec)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:41:14.130857 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4f9c93-cbd6-47c1-a0fb-bc79e75d1372 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:41:14.177940 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-db4f9c93-cbd6-47c1-a0fb-bc79e75d1372 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ed115ccc00604c3cb1fb3767d5dd3cec', 'request_id': 'req-db4f9c93-cbd6-47c1-a0fb-bc79e75d1372'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:41:14.184443 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-db4f9c93-cbd6-47c1-a0fb-bc79e75d1372 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ed115ccc00604c3cb1fb3767d5dd3cec', 'request_id': 'req-db4f9c93-cbd6-47c1-a0fb-bc79e75d1372'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:41:14.192366 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-db4f9c93-cbd6-47c1-a0fb-bc79e75d1372 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ed115ccc00604c3cb1fb3767d5dd3cec', 'request_id': 'req-db4f9c93-cbd6-47c1-a0fb-bc79e75d1372'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:41:14.202814 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-db4f9c93-cbd6-47c1-a0fb-bc79e75d1372 admin admin] The token cache is being invalidate because project ed115ccc00604c3cb1fb3767d5dd3cec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81501) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:41:14.203099 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-db4f9c93-cbd6-47c1-a0fb-bc79e75d1372 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:41:14.203796 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-db4f9c93-cbd6-47c1-a0fb-bc79e75d1372 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:41:14.205315 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1235/2488] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Nov 2 07:41:14 2022] DELETE /identity/v3/projects/ed115ccc00604c3cb1fb3767d5dd3cec => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:41:14.210147 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8419a49d-1fce-4934-89f6-865a420ef49a None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:41:14.210667 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8419a49d-1fce-4934-89f6-865a420ef49a None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:41:14.210786 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8419a49d-1fce-4934-89f6-865a420ef49a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:41:14.231965 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-8419a49d-1fce-4934-89f6-865a420ef49a None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:41:14.233167 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-8419a49d-1fce-4934-89f6-865a420ef49a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:41:14.290610 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1254/2489] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:41:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:41:14.299203 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e477c8-1abe-479d-93ad-ad54a4f47e72 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:41:14.299493 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e477c8-1abe-479d-93ad-ad54a4f47e72 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:41:14.299770 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e477c8-1abe-479d-93ad-ad54a4f47e72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:41:14.319092 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-70e477c8-1abe-479d-93ad-ad54a4f47e72 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:41:14.320284 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-70e477c8-1abe-479d-93ad-ad54a4f47e72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:41:14.329003 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1236/2490] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:41:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:41:14.333296 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8419a49d-1fce-4934-89f6-865a420ef49a None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:41:14.343639 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8419a49d-1fce-4934-89f6-865a420ef49a None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:41:14.348513 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a101d406-0528-48e5-b8e7-c1b569bd1eb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:41:14.349111 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a101d406-0528-48e5-b8e7-c1b569bd1eb2 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:41:14.349391 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a101d406-0528-48e5-b8e7-c1b569bd1eb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:41:14.349664 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a101d406-0528-48e5-b8e7-c1b569bd1eb2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:41:14.350309 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a101d406-0528-48e5-b8e7-c1b569bd1eb2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:41:14.350634 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a101d406-0528-48e5-b8e7-c1b569bd1eb2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:41:14.352237 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a101d406-0528-48e5-b8e7-c1b569bd1eb2 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:41:14.364273 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1255/2491] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Nov 2 07:41:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:41:14.368534 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e477c8-1abe-479d-93ad-ad54a4f47e72 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:41:14.377770 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e477c8-1abe-479d-93ad-ad54a4f47e72 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:41:14.382542 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71709f9-dcbf-43ab-b362-b905a598aa3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:41:14.383171 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71709f9-dcbf-43ab-b362-b905a598aa3c admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:41:14.383455 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71709f9-dcbf-43ab-b362-b905a598aa3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:41:14.383726 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71709f9-dcbf-43ab-b362-b905a598aa3c admin admin] PATH_INFO: `/v3/projects` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:41:14.384373 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71709f9-dcbf-43ab-b362-b905a598aa3c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:41:14.386789 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71709f9-dcbf-43ab-b362-b905a598aa3c admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:41:14.415171 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1237/2492] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Nov 2 07:41:14 2022] POST /identity/v3/projects => generated 377 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:41:14.419327 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a101d406-0528-48e5-b8e7-c1b569bd1eb2 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:41:14.429387 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a101d406-0528-48e5-b8e7-c1b569bd1eb2 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:41:14.449419 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:41:14.487285 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c2b61c-2181-4556-b857-2f054cc2ffba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:41:14.487881 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c2b61c-2181-4556-b857-2f054cc2ffba admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:41:14.488161 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c2b61c-2181-4556-b857-2f054cc2ffba admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:41:14.488431 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c2b61c-2181-4556-b857-2f054cc2ffba admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:41:14.489058 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c2b61c-2181-4556-b857-2f054cc2ffba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:41:14.490990 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c2b61c-2181-4556-b857-2f054cc2ffba admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:41:14.512446 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1256/2493] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Nov 2 07:41:14 2022] POST /identity/v3/users => generated 289 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:41:14.516784 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71709f9-dcbf-43ab-b362-b905a598aa3c admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:41:14.527713 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71709f9-dcbf-43ab-b362-b905a598aa3c admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:41:14.532314 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4f7816-22d0-440b-b7d1-7f39eee4a85b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:41:14.532903 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4f7816-22d0-440b-b7d1-7f39eee4a85b admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:41:14.533181 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4f7816-22d0-440b-b7d1-7f39eee4a85b admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:41:14.533454 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4f7816-22d0-440b-b7d1-7f39eee4a85b admin admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:41:14.534083 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4f7816-22d0-440b-b7d1-7f39eee4a85b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:41:14.535689 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4f7816-22d0-440b-b7d1-7f39eee4a85b admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:41:14.545472 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1238/2494] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:41:14 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:41:14.549644 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c2b61c-2181-4556-b857-2f054cc2ffba admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:41:14.559759 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c2b61c-2181-4556-b857-2f054cc2ffba admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:41:14.564262 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da25061f-dd43-4dfe-a8a6-bde65688c816 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:41:14.564845 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da25061f-dd43-4dfe-a8a6-bde65688c816 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:41:14.565121 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da25061f-dd43-4dfe-a8a6-bde65688c816 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:41:14.565389 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da25061f-dd43-4dfe-a8a6-bde65688c816 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:41:14.566016 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da25061f-dd43-4dfe-a8a6-bde65688c816 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:41:14.567632 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da25061f-dd43-4dfe-a8a6-bde65688c816 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:41:14.578453 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1257/2495] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:41:14 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:41:14.582444 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4f7816-22d0-440b-b7d1-7f39eee4a85b admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:41:14.592730 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4f7816-22d0-440b-b7d1-7f39eee4a85b admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:41:14.597471 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0d3258-7447-41ef-a513-d0cb2541bc4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:41:14.598100 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0d3258-7447-41ef-a513-d0cb2541bc4d admin admin] REQUEST_METHOD: `PUT` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:41:14.598400 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0d3258-7447-41ef-a513-d0cb2541bc4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:41:14.598849 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0d3258-7447-41ef-a513-d0cb2541bc4d admin admin] PATH_INFO: `/v3/projects/f625974081b1421e85e1a9bfc43799c5/users/113df3ade84f4b6f8549708303d627d5/roles/b494fedd00f24f0b84a529e8672a156a` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:41:14.613285 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0d3258-7447-41ef-a513-d0cb2541bc4d admin admin] RBAC: Authorizing `identity:create_grant(project_id=f625974081b1421e85e1a9bfc43799c5, user_id=113df3ade84f4b6f8549708303d627d5, role_id=b494fedd00f24f0b84a529e8672a156a)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:41:14.615912 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0d3258-7447-41ef-a513-d0cb2541bc4d admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:41:14.628328 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1239/2496] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Nov 2 07:41:14 2022] PUT /identity/v3/projects/f625974081b1421e85e1a9bfc43799c5/users/113df3ade84f4b6f8549708303d627d5/roles/b494fedd00f24f0b84a529e8672a156a => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:41:14.640498 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cdb0da-b982-45c7-bfef-252a14b58094 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:41:14.640799 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cdb0da-b982-45c7-bfef-252a14b58094 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:41:14.641074 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cdb0da-b982-45c7-bfef-252a14b58094 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:41:14.668922 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-48cdb0da-b982-45c7-bfef-252a14b58094 None None] MFA Rules not processed for user `113df3ade84f4b6f8549708303d627d5`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:41:14.670126 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-48cdb0da-b982-45c7-bfef-252a14b58094 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:41:14.732474 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1258/2497] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:41:14 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:41:14.737371 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0d3258-7447-41ef-a513-d0cb2541bc4d admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:41:14.784950 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:41:14.799980 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0d3258-7447-41ef-a513-d0cb2541bc4d admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:41:14.804700 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f73187-e783-4d17-aa29-8877d4da2930 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:41:14.805301 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f73187-e783-4d17-aa29-8877d4da2930 admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:41:14.805580 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f73187-e783-4d17-aa29-8877d4da2930 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:41:14.805877 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f73187-e783-4d17-aa29-8877d4da2930 admin admin] PATH_INFO: `/v3/projects` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:41:14.806530 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f73187-e783-4d17-aa29-8877d4da2930 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:41:14.808959 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f73187-e783-4d17-aa29-8877d4da2930 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:41:14.834489 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1240/2498] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Nov 2 07:41:14 2022] POST /identity/v3/projects => generated 375 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:41:14.838717 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cdb0da-b982-45c7-bfef-252a14b58094 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:41:14.849415 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cdb0da-b982-45c7-bfef-252a14b58094 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:41:14.904208 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ae3995-964e-43c1-985d-4146697ad26d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:41:14.904808 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ae3995-964e-43c1-985d-4146697ad26d admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:41:14.905089 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ae3995-964e-43c1-985d-4146697ad26d admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:41:14.905357 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ae3995-964e-43c1-985d-4146697ad26d admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:41:14.905988 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ae3995-964e-43c1-985d-4146697ad26d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:41:14.907909 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ae3995-964e-43c1-985d-4146697ad26d admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:41:14.928947 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1259/2499] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Nov 2 07:41:14 2022] POST /identity/v3/users => generated 294 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:41:14.933085 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f73187-e783-4d17-aa29-8877d4da2930 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:41:14.943342 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f73187-e783-4d17-aa29-8877d4da2930 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:41:14.948247 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93f1460-56f9-4157-8b8e-9ca4771854f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:41:14.948968 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93f1460-56f9-4157-8b8e-9ca4771854f6 admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:41:14.949273 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93f1460-56f9-4157-8b8e-9ca4771854f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:41:14.949705 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93f1460-56f9-4157-8b8e-9ca4771854f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:41:14.950151 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93f1460-56f9-4157-8b8e-9ca4771854f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:41:14.951977 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93f1460-56f9-4157-8b8e-9ca4771854f6 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:41:14.962433 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1241/2500] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:41:14 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:41:14.967220 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ae3995-964e-43c1-985d-4146697ad26d admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:41:14.976969 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ae3995-964e-43c1-985d-4146697ad26d admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:41:14.982677 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf204b9-cf58-4826-a598-6d3dd80e879c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:41:14.983838 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf204b9-cf58-4826-a598-6d3dd80e879c admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:41:14.984137 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf204b9-cf58-4826-a598-6d3dd80e879c admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:41:14.984413 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf204b9-cf58-4826-a598-6d3dd80e879c admin admin] PATH_INFO: `/v3/projects/02c17c6697174484ac76bb8dab25c0f8/users/48bb8a859d0c4802bf91b17b0a0cbeae/roles/c4bf019fc6c44c71a6c984f6d0bab75f` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:41:14.998970 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf204b9-cf58-4826-a598-6d3dd80e879c admin admin] RBAC: Authorizing `identity:create_grant(project_id=02c17c6697174484ac76bb8dab25c0f8, user_id=48bb8a859d0c4802bf91b17b0a0cbeae, role_id=c4bf019fc6c44c71a6c984f6d0bab75f)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:41:15.001578 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf204b9-cf58-4826-a598-6d3dd80e879c admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:41:15.009463 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:41:15.016187 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1260/2501] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Nov 2 07:41:14 2022] PUT /identity/v3/projects/02c17c6697174484ac76bb8dab25c0f8/users/48bb8a859d0c4802bf91b17b0a0cbeae/roles/c4bf019fc6c44c71a6c984f6d0bab75f => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:41:15.020331 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93f1460-56f9-4157-8b8e-9ca4771854f6 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:41:15.031868 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93f1460-56f9-4157-8b8e-9ca4771854f6 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:41:15.087633 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534dca52-7379-415d-81a8-5912a733ed31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:41:15.088237 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534dca52-7379-415d-81a8-5912a733ed31 admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:41:15.088518 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534dca52-7379-415d-81a8-5912a733ed31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:41:15.088789 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534dca52-7379-415d-81a8-5912a733ed31 admin admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:41:15.090520 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534dca52-7379-415d-81a8-5912a733ed31 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:41:15.092115 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534dca52-7379-415d-81a8-5912a733ed31 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:41:15.101522 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1242/2502] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:41:15 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:41:15.105937 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf204b9-cf58-4826-a598-6d3dd80e879c admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:41:15.116227 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf204b9-cf58-4826-a598-6d3dd80e879c admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:41:15.120838 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041771ca-8811-4d0c-ad81-42d1e30d0c7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:41:15.121476 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041771ca-8811-4d0c-ad81-42d1e30d0c7d admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:41:15.121752 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041771ca-8811-4d0c-ad81-42d1e30d0c7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:41:15.122020 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041771ca-8811-4d0c-ad81-42d1e30d0c7d admin admin] PATH_INFO: `/v3/projects/02c17c6697174484ac76bb8dab25c0f8/users/48bb8a859d0c4802bf91b17b0a0cbeae/roles/c4bf019fc6c44c71a6c984f6d0bab75f` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:41:15.123954 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041771ca-8811-4d0c-ad81-42d1e30d0c7d admin admin] RBAC: Authorizing `identity:create_grant(project_id=02c17c6697174484ac76bb8dab25c0f8, user_id=48bb8a859d0c4802bf91b17b0a0cbeae, role_id=c4bf019fc6c44c71a6c984f6d0bab75f)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:41:15.126562 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041771ca-8811-4d0c-ad81-42d1e30d0c7d admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:41:15.137250 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1261/2503] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Nov 2 07:41:15 2022] PUT /identity/v3/projects/02c17c6697174484ac76bb8dab25c0f8/users/48bb8a859d0c4802bf91b17b0a0cbeae/roles/c4bf019fc6c44c71a6c984f6d0bab75f => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:41:15.147718 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e28811-8c47-4388-9183-04dd05c22e9a None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:41:15.148003 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e28811-8c47-4388-9183-04dd05c22e9a None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:41:15.148278 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e28811-8c47-4388-9183-04dd05c22e9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:41:15.176982 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-f6e28811-8c47-4388-9183-04dd05c22e9a None None] MFA Rules not processed for user `48bb8a859d0c4802bf91b17b0a0cbeae`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:41:15.178184 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-f6e28811-8c47-4388-9183-04dd05c22e9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:41:15.244800 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1243/2504] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:41:15 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:41:15.255858 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041771ca-8811-4d0c-ad81-42d1e30d0c7d admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:41:15.257626 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-041771ca-8811-4d0c-ad81-42d1e30d0c7d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:41:15.315434 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041771ca-8811-4d0c-ad81-42d1e30d0c7d admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:41:15.319096 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10065548-5be4-4721-8d5e-0febd8e32e2f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43bb326fb82e4d2c83fb2eaf91860dc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:41:15.319699 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10065548-5be4-4721-8d5e-0febd8e32e2f service cinder] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:41:15.319999 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10065548-5be4-4721-8d5e-0febd8e32e2f service cinder] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:41:15.320252 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10065548-5be4-4721-8d5e-0febd8e32e2f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:41:15.377629 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10065548-5be4-4721-8d5e-0febd8e32e2f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:41:15.379089 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10065548-5be4-4721-8d5e-0febd8e32e2f service cinder] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:41:15.388739 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1262/2505] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:41:15 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:41:20.016104 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e28811-8c47-4388-9183-04dd05c22e9a None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:41:20.025693 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e28811-8c47-4388-9183-04dd05c22e9a None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:41:20.029153 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f54cc83-3878-4d72-bd46-c5fa3e0526cd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43bb326fb82e4d2c83fb2eaf91860dc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:41:20.029771 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f54cc83-3878-4d72-bd46-c5fa3e0526cd service cinder] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:41:20.030121 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f54cc83-3878-4d72-bd46-c5fa3e0526cd service cinder] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:41:20.030458 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f54cc83-3878-4d72-bd46-c5fa3e0526cd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:41:20.039277 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f54cc83-3878-4d72-bd46-c5fa3e0526cd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:41:20.040717 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f54cc83-3878-4d72-bd46-c5fa3e0526cd service cinder] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:41:20.049826 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1244/2506] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:41:20 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:41:25.384311 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=54,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:41:30.045777 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=86,keystone:INSERT=1 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:41:42.168987 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10065548-5be4-4721-8d5e-0febd8e32e2f service cinder] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:41:42.231087 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10065548-5be4-4721-8d5e-0febd8e32e2f service cinder] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:41:42.235835 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d0ebdf-8250-48b4-a1f5-18aaa3c66f9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:41:42.236483 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d0ebdf-8250-48b4-a1f5-18aaa3c66f9f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:41:42.236760 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d0ebdf-8250-48b4-a1f5-18aaa3c66f9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:41:42.237026 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d0ebdf-8250-48b4-a1f5-18aaa3c66f9f admin admin] PATH_INFO: `/v3/users/113df3ade84f4b6f8549708303d627d5` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:41:42.238154 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d0ebdf-8250-48b4-a1f5-18aaa3c66f9f admin admin] RBAC: Authorizing `identity:delete_user(user_id=113df3ade84f4b6f8549708303d627d5)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:41:42.240108 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d0ebdf-8250-48b4-a1f5-18aaa3c66f9f admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:41:42.285973 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-64d0ebdf-8250-48b4-a1f5-18aaa3c66f9f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '113df3ade84f4b6f8549708303d627d5'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:41:42.293156 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-64d0ebdf-8250-48b4-a1f5-18aaa3c66f9f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '113df3ade84f4b6f8549708303d627d5'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:41:42.293767 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-64d0ebdf-8250-48b4-a1f5-18aaa3c66f9f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '113df3ade84f4b6f8549708303d627d5'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:41:42.301171 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-64d0ebdf-8250-48b4-a1f5-18aaa3c66f9f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '113df3ade84f4b6f8549708303d627d5'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:41:42.301854 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-64d0ebdf-8250-48b4-a1f5-18aaa3c66f9f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '113df3ade84f4b6f8549708303d627d5'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:41:42.319504 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1263/2507] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Nov 2 07:41:42 2022] DELETE /identity/v3/users/113df3ade84f4b6f8549708303d627d5 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:41:42.324043 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19ef676-50b0-4ae1-b0cf-73661740ecc0 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:41:42.324512 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19ef676-50b0-4ae1-b0cf-73661740ecc0 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:41:42.324861 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19ef676-50b0-4ae1-b0cf-73661740ecc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:41:42.346157 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-e19ef676-50b0-4ae1-b0cf-73661740ecc0 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:41:42.347530 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-e19ef676-50b0-4ae1-b0cf-73661740ecc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:41:42.408926 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1245/2508] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:41:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:41:42.418451 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d0ebdf-8250-48b4-a1f5-18aaa3c66f9f admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:41:42.419860 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-64d0ebdf-8250-48b4-a1f5-18aaa3c66f9f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:41:42.475991 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d0ebdf-8250-48b4-a1f5-18aaa3c66f9f admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:41:42.479664 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36892237-5889-48b0-829f-b65f7ac34e80 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3ec6ecb85f94ef59762877d9ea33ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:41:42.480256 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36892237-5889-48b0-829f-b65f7ac34e80 service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:41:42.480535 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36892237-5889-48b0-829f-b65f7ac34e80 service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:41:42.480804 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36892237-5889-48b0-829f-b65f7ac34e80 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:41:42.489671 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36892237-5889-48b0-829f-b65f7ac34e80 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:41:42.491134 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36892237-5889-48b0-829f-b65f7ac34e80 service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:41:42.501100 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1264/2509] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:41:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:41:42.786967 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19ef676-50b0-4ae1-b0cf-73661740ecc0 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:41:42.788558 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-e19ef676-50b0-4ae1-b0cf-73661740ecc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:41:42.801747 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19ef676-50b0-4ae1-b0cf-73661740ecc0 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:41:42.806533 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf73602-c035-4fc9-8aed-9b2dec4cb6a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:41:42.807167 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf73602-c035-4fc9-8aed-9b2dec4cb6a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:41:42.807451 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf73602-c035-4fc9-8aed-9b2dec4cb6a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:41:42.807722 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf73602-c035-4fc9-8aed-9b2dec4cb6a9 admin admin] PATH_INFO: `/v3/projects/f625974081b1421e85e1a9bfc43799c5` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:41:42.808837 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf73602-c035-4fc9-8aed-9b2dec4cb6a9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f625974081b1421e85e1a9bfc43799c5)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:41:42.810794 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf73602-c035-4fc9-8aed-9b2dec4cb6a9 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:41:42.858716 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-3bf73602-c035-4fc9-8aed-9b2dec4cb6a9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f625974081b1421e85e1a9bfc43799c5', 'request_id': 'req-3bf73602-c035-4fc9-8aed-9b2dec4cb6a9'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:41:42.865903 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-3bf73602-c035-4fc9-8aed-9b2dec4cb6a9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f625974081b1421e85e1a9bfc43799c5', 'request_id': 'req-3bf73602-c035-4fc9-8aed-9b2dec4cb6a9'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:41:42.873601 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-3bf73602-c035-4fc9-8aed-9b2dec4cb6a9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f625974081b1421e85e1a9bfc43799c5', 'request_id': 'req-3bf73602-c035-4fc9-8aed-9b2dec4cb6a9'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:41:42.884961 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-3bf73602-c035-4fc9-8aed-9b2dec4cb6a9 admin admin] The token cache is being invalidate because project f625974081b1421e85e1a9bfc43799c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81501) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:41:42.885273 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-3bf73602-c035-4fc9-8aed-9b2dec4cb6a9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:41:42.885926 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-3bf73602-c035-4fc9-8aed-9b2dec4cb6a9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:41:42.887424 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1246/2510] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Nov 2 07:41:42 2022] DELETE /identity/v3/projects/f625974081b1421e85e1a9bfc43799c5 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:41:42.891466 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36892237-5889-48b0-829f-b65f7ac34e80 service neutron] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:41:42.901507 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36892237-5889-48b0-829f-b65f7ac34e80 service neutron] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:41:42.921344 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:41:42.962311 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6663ba70-ee57-46a3-8666-b14c4ea045a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:41:42.962986 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6663ba70-ee57-46a3-8666-b14c4ea045a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:41:42.963263 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6663ba70-ee57-46a3-8666-b14c4ea045a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:41:42.963533 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6663ba70-ee57-46a3-8666-b14c4ea045a3 admin admin] PATH_INFO: `/v3/users/48bb8a859d0c4802bf91b17b0a0cbeae` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:41:42.964622 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6663ba70-ee57-46a3-8666-b14c4ea045a3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=48bb8a859d0c4802bf91b17b0a0cbeae)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:41:42.967151 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6663ba70-ee57-46a3-8666-b14c4ea045a3 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:41:43.015176 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-6663ba70-ee57-46a3-8666-b14c4ea045a3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '48bb8a859d0c4802bf91b17b0a0cbeae'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:41:43.022753 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-6663ba70-ee57-46a3-8666-b14c4ea045a3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '48bb8a859d0c4802bf91b17b0a0cbeae'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:41:43.023362 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-6663ba70-ee57-46a3-8666-b14c4ea045a3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '48bb8a859d0c4802bf91b17b0a0cbeae'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:41:43.030171 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-6663ba70-ee57-46a3-8666-b14c4ea045a3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '48bb8a859d0c4802bf91b17b0a0cbeae'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:41:43.030825 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-6663ba70-ee57-46a3-8666-b14c4ea045a3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '48bb8a859d0c4802bf91b17b0a0cbeae'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:41:43.048392 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1265/2511] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Nov 2 07:41:42 2022] DELETE /identity/v3/users/48bb8a859d0c4802bf91b17b0a0cbeae => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:41:43.321607 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf73602-c035-4fc9-8aed-9b2dec4cb6a9 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:41:43.323095 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-3bf73602-c035-4fc9-8aed-9b2dec4cb6a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:41:43.340338 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf73602-c035-4fc9-8aed-9b2dec4cb6a9 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:41:43.402567 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22f39a2-5468-4be8-83a9-cc21a24c8e61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:41:43.403242 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22f39a2-5468-4be8-83a9-cc21a24c8e61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:41:43.403521 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22f39a2-5468-4be8-83a9-cc21a24c8e61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:41:43.403792 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22f39a2-5468-4be8-83a9-cc21a24c8e61 admin admin] PATH_INFO: `/v3/projects/02c17c6697174484ac76bb8dab25c0f8` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:41:43.404964 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22f39a2-5468-4be8-83a9-cc21a24c8e61 admin admin] RBAC: Authorizing `identity:delete_project(project_id=02c17c6697174484ac76bb8dab25c0f8)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:41:43.406902 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22f39a2-5468-4be8-83a9-cc21a24c8e61 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:41:43.438790 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:41:43.457644 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-b22f39a2-5468-4be8-83a9-cc21a24c8e61 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '02c17c6697174484ac76bb8dab25c0f8', 'request_id': 'req-b22f39a2-5468-4be8-83a9-cc21a24c8e61'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:41:43.463696 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-b22f39a2-5468-4be8-83a9-cc21a24c8e61 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '02c17c6697174484ac76bb8dab25c0f8', 'request_id': 'req-b22f39a2-5468-4be8-83a9-cc21a24c8e61'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:41:43.471710 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-b22f39a2-5468-4be8-83a9-cc21a24c8e61 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '02c17c6697174484ac76bb8dab25c0f8', 'request_id': 'req-b22f39a2-5468-4be8-83a9-cc21a24c8e61'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:41:43.482703 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-b22f39a2-5468-4be8-83a9-cc21a24c8e61 admin admin] The token cache is being invalidate because project 02c17c6697174484ac76bb8dab25c0f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81501) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:41:43.482994 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-b22f39a2-5468-4be8-83a9-cc21a24c8e61 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:41:43.483627 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-b22f39a2-5468-4be8-83a9-cc21a24c8e61 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:41:43.485874 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1247/2512] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Nov 2 07:41:43 2022] DELETE /identity/v3/projects/02c17c6697174484ac76bb8dab25c0f8 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:41:43.491257 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36884759-8c40-4bef-9472-a846d890e3ca None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:41:43.491796 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36884759-8c40-4bef-9472-a846d890e3ca None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:41:43.491796 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36884759-8c40-4bef-9472-a846d890e3ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:41:43.513454 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-36884759-8c40-4bef-9472-a846d890e3ca None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:41:43.514855 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-36884759-8c40-4bef-9472-a846d890e3ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:41:43.574347 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1266/2513] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:41:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:41:43.583319 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b96b6c-5833-4d54-b56b-fd6508e859c3 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:41:43.583611 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b96b6c-5833-4d54-b56b-fd6508e859c3 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:41:43.583888 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b96b6c-5833-4d54-b56b-fd6508e859c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:41:43.604299 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-52b96b6c-5833-4d54-b56b-fd6508e859c3 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:41:43.605538 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-52b96b6c-5833-4d54-b56b-fd6508e859c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:41:43.613880 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1248/2514] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:41:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:41:43.618079 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36884759-8c40-4bef-9472-a846d890e3ca None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:41:43.628145 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36884759-8c40-4bef-9472-a846d890e3ca None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:41:43.632990 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b492eae-7c12-4caf-9669-45d62b9b12c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:41:43.633560 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b492eae-7c12-4caf-9669-45d62b9b12c0 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:41:43.633840 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b492eae-7c12-4caf-9669-45d62b9b12c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:41:43.634108 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b492eae-7c12-4caf-9669-45d62b9b12c0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:41:43.634789 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b492eae-7c12-4caf-9669-45d62b9b12c0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:41:43.635086 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b492eae-7c12-4caf-9669-45d62b9b12c0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:41:43.636652 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b492eae-7c12-4caf-9669-45d62b9b12c0 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:41:43.648303 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1267/2515] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Nov 2 07:41:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:41:43.652599 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b96b6c-5833-4d54-b56b-fd6508e859c3 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:41:43.662657 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b96b6c-5833-4d54-b56b-fd6508e859c3 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:41:43.667319 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2680dde5-adac-4033-83db-4fd764ee4302 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:41:43.667904 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2680dde5-adac-4033-83db-4fd764ee4302 admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:41:43.668183 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2680dde5-adac-4033-83db-4fd764ee4302 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:41:43.668451 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2680dde5-adac-4033-83db-4fd764ee4302 admin admin] PATH_INFO: `/v3/projects` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:41:43.669069 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2680dde5-adac-4033-83db-4fd764ee4302 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:41:43.670981 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2680dde5-adac-4033-83db-4fd764ee4302 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:41:43.698865 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1249/2516] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Nov 2 07:41:43 2022] POST /identity/v3/projects => generated 385 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:41:43.703024 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b492eae-7c12-4caf-9669-45d62b9b12c0 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:41:43.713304 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b492eae-7c12-4caf-9669-45d62b9b12c0 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:41:43.739753 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:41:43.771597 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6259e7d-73d4-4694-8e0e-39347e543d5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:41:43.772205 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6259e7d-73d4-4694-8e0e-39347e543d5e admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:41:43.772485 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6259e7d-73d4-4694-8e0e-39347e543d5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:41:43.772759 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6259e7d-73d4-4694-8e0e-39347e543d5e admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:41:43.773384 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6259e7d-73d4-4694-8e0e-39347e543d5e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:41:43.775384 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6259e7d-73d4-4694-8e0e-39347e543d5e admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:41:43.796685 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1268/2517] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Nov 2 07:41:43 2022] POST /identity/v3/users => generated 293 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:41:43.801981 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2680dde5-adac-4033-83db-4fd764ee4302 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:41:43.812081 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2680dde5-adac-4033-83db-4fd764ee4302 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:41:43.817670 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d21aee-d9de-4a2a-82a2-4476dc949a05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:41:43.818260 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d21aee-d9de-4a2a-82a2-4476dc949a05 admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:41:43.818595 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d21aee-d9de-4a2a-82a2-4476dc949a05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:41:43.818850 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d21aee-d9de-4a2a-82a2-4476dc949a05 admin admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:41:43.819482 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d21aee-d9de-4a2a-82a2-4476dc949a05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:41:43.821062 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d21aee-d9de-4a2a-82a2-4476dc949a05 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:41:43.831831 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1250/2518] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:41:43 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:41:43.835670 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6259e7d-73d4-4694-8e0e-39347e543d5e admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:41:43.845894 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6259e7d-73d4-4694-8e0e-39347e543d5e admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:41:43.850497 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95ced49-2a32-4ffe-bf01-aed4685ce358 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:41:43.851080 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95ced49-2a32-4ffe-bf01-aed4685ce358 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:41:43.851355 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95ced49-2a32-4ffe-bf01-aed4685ce358 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:41:43.851620 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95ced49-2a32-4ffe-bf01-aed4685ce358 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:41:43.852336 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95ced49-2a32-4ffe-bf01-aed4685ce358 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:41:43.854065 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95ced49-2a32-4ffe-bf01-aed4685ce358 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:41:43.865288 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1269/2519] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:41:43 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:41:43.869409 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d21aee-d9de-4a2a-82a2-4476dc949a05 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:41:43.880156 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d21aee-d9de-4a2a-82a2-4476dc949a05 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:41:43.885038 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf0646b-2fde-440e-93b6-ad88907f2b05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:41:43.885675 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf0646b-2fde-440e-93b6-ad88907f2b05 admin admin] REQUEST_METHOD: `PUT` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:41:43.885955 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf0646b-2fde-440e-93b6-ad88907f2b05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:41:43.886223 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf0646b-2fde-440e-93b6-ad88907f2b05 admin admin] PATH_INFO: `/v3/projects/f4b31ed293214e9d91670bef92f6b84a/users/c94866c100a04a8791ca5eb5047f9c9d/roles/b494fedd00f24f0b84a529e8672a156a` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:41:43.901483 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf0646b-2fde-440e-93b6-ad88907f2b05 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4b31ed293214e9d91670bef92f6b84a, user_id=c94866c100a04a8791ca5eb5047f9c9d, role_id=b494fedd00f24f0b84a529e8672a156a)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:41:43.904154 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf0646b-2fde-440e-93b6-ad88907f2b05 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:41:43.917771 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1251/2520] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Nov 2 07:41:43 2022] PUT /identity/v3/projects/f4b31ed293214e9d91670bef92f6b84a/users/c94866c100a04a8791ca5eb5047f9c9d/roles/b494fedd00f24f0b84a529e8672a156a => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:41:43.928602 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ba06fa-39cb-4efa-a79d-3bc1d8bfd72e None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:41:43.928929 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ba06fa-39cb-4efa-a79d-3bc1d8bfd72e None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:41:43.929233 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ba06fa-39cb-4efa-a79d-3bc1d8bfd72e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:41:43.958347 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-89ba06fa-39cb-4efa-a79d-3bc1d8bfd72e None None] MFA Rules not processed for user `c94866c100a04a8791ca5eb5047f9c9d`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:41:43.959626 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-89ba06fa-39cb-4efa-a79d-3bc1d8bfd72e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:41:44.021708 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1270/2521] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:41:43 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:41:44.026402 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf0646b-2fde-440e-93b6-ad88907f2b05 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:41:44.090960 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf0646b-2fde-440e-93b6-ad88907f2b05 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:41:44.095625 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73bf51b-5f51-4a57-9842-d9d94e5598f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:41:44.096243 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73bf51b-5f51-4a57-9842-d9d94e5598f9 admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:41:44.096517 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73bf51b-5f51-4a57-9842-d9d94e5598f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:41:44.096784 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73bf51b-5f51-4a57-9842-d9d94e5598f9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:41:44.097398 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73bf51b-5f51-4a57-9842-d9d94e5598f9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:41:44.099309 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73bf51b-5f51-4a57-9842-d9d94e5598f9 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:41:44.127581 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1252/2522] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Nov 2 07:41:44 2022] POST /identity/v3/projects => generated 387 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:41:44.132485 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ba06fa-39cb-4efa-a79d-3bc1d8bfd72e None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:41:44.143706 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ba06fa-39cb-4efa-a79d-3bc1d8bfd72e None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:41:44.200546 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4c35d9-8a9a-47f3-9710-09a9be4e9aeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:41:44.201266 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4c35d9-8a9a-47f3-9710-09a9be4e9aeb admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:41:44.201593 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4c35d9-8a9a-47f3-9710-09a9be4e9aeb admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:41:44.201864 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4c35d9-8a9a-47f3-9710-09a9be4e9aeb admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:41:44.202583 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4c35d9-8a9a-47f3-9710-09a9be4e9aeb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:41:44.204537 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4c35d9-8a9a-47f3-9710-09a9be4e9aeb admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:41:44.225215 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1271/2523] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Nov 2 07:41:44 2022] POST /identity/v3/users => generated 300 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:41:44.229259 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73bf51b-5f51-4a57-9842-d9d94e5598f9 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:41:44.240290 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73bf51b-5f51-4a57-9842-d9d94e5598f9 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:41:44.244766 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb8d0b8-0dbd-43a2-9264-1fa3819bef87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:41:44.245352 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb8d0b8-0dbd-43a2-9264-1fa3819bef87 admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:41:44.245644 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb8d0b8-0dbd-43a2-9264-1fa3819bef87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:41:44.245900 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb8d0b8-0dbd-43a2-9264-1fa3819bef87 admin admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:41:44.246561 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb8d0b8-0dbd-43a2-9264-1fa3819bef87 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:41:44.248137 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb8d0b8-0dbd-43a2-9264-1fa3819bef87 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:41:44.261428 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1253/2524] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:41:44 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:41:44.265772 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4c35d9-8a9a-47f3-9710-09a9be4e9aeb admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:41:44.275810 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4c35d9-8a9a-47f3-9710-09a9be4e9aeb admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:41:44.280261 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c811cb04-af14-4921-a20c-a906cabd72bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:41:44.280905 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c811cb04-af14-4921-a20c-a906cabd72bb admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:41:44.281180 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c811cb04-af14-4921-a20c-a906cabd72bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:41:44.281448 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c811cb04-af14-4921-a20c-a906cabd72bb admin admin] PATH_INFO: `/v3/projects/03232af795254812ab229b91b649176e/users/e4041b0193414bc08dce62e848ddf308/roles/c4bf019fc6c44c71a6c984f6d0bab75f` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:41:44.296749 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c811cb04-af14-4921-a20c-a906cabd72bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=03232af795254812ab229b91b649176e, user_id=e4041b0193414bc08dce62e848ddf308, role_id=c4bf019fc6c44c71a6c984f6d0bab75f)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:41:44.300426 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c811cb04-af14-4921-a20c-a906cabd72bb admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:41:44.312962 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1272/2525] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Nov 2 07:41:44 2022] PUT /identity/v3/projects/03232af795254812ab229b91b649176e/users/e4041b0193414bc08dce62e848ddf308/roles/c4bf019fc6c44c71a6c984f6d0bab75f => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 07:41:44.317711 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb8d0b8-0dbd-43a2-9264-1fa3819bef87 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:41:44.327547 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb8d0b8-0dbd-43a2-9264-1fa3819bef87 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:41:44.346868 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:41:44.387491 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887754db-bf9e-4c08-938a-c1f7f650b6f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:41:44.388105 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887754db-bf9e-4c08-938a-c1f7f650b6f9 admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:41:44.388388 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887754db-bf9e-4c08-938a-c1f7f650b6f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:41:44.388791 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887754db-bf9e-4c08-938a-c1f7f650b6f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:41:44.389441 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887754db-bf9e-4c08-938a-c1f7f650b6f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:41:44.391072 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887754db-bf9e-4c08-938a-c1f7f650b6f9 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:41:44.401066 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1254/2526] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:41:44 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 07:41:44.405741 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c811cb04-af14-4921-a20c-a906cabd72bb admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:41:44.412136 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:41:44.418810 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c811cb04-af14-4921-a20c-a906cabd72bb admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:41:44.424673 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48012b8-01af-4670-8bfd-6f91243501eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:41:44.425330 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48012b8-01af-4670-8bfd-6f91243501eb admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:41:44.425608 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48012b8-01af-4670-8bfd-6f91243501eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:41:44.425891 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48012b8-01af-4670-8bfd-6f91243501eb admin admin] PATH_INFO: `/v3/projects/03232af795254812ab229b91b649176e/users/e4041b0193414bc08dce62e848ddf308/roles/c4bf019fc6c44c71a6c984f6d0bab75f` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:41:44.427861 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48012b8-01af-4670-8bfd-6f91243501eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=03232af795254812ab229b91b649176e, user_id=e4041b0193414bc08dce62e848ddf308, role_id=c4bf019fc6c44c71a6c984f6d0bab75f)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:41:44.430575 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48012b8-01af-4670-8bfd-6f91243501eb admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:41:44.440707 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1273/2527] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Nov 2 07:41:44 2022] PUT /identity/v3/projects/03232af795254812ab229b91b649176e/users/e4041b0193414bc08dce62e848ddf308/roles/c4bf019fc6c44c71a6c984f6d0bab75f => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:41:44.450855 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce09a20-9b33-4502-a473-14a101ee7d2f None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:41:44.451146 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce09a20-9b33-4502-a473-14a101ee7d2f None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:41:44.451424 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce09a20-9b33-4502-a473-14a101ee7d2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:41:44.479599 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-4ce09a20-9b33-4502-a473-14a101ee7d2f None None] MFA Rules not processed for user `e4041b0193414bc08dce62e848ddf308`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:41:44.480799 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-4ce09a20-9b33-4502-a473-14a101ee7d2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:41:44.547222 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1255/2528] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:41:44 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:41:44.558925 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48012b8-01af-4670-8bfd-6f91243501eb admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:41:44.560605 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-e48012b8-01af-4670-8bfd-6f91243501eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:41:44.617149 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48012b8-01af-4670-8bfd-6f91243501eb admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:41:44.620809 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf92c6d6-40f3-4bff-9190-c48e2bbb4c27 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43bb326fb82e4d2c83fb2eaf91860dc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:41:44.621401 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf92c6d6-40f3-4bff-9190-c48e2bbb4c27 service cinder] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:41:44.621677 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf92c6d6-40f3-4bff-9190-c48e2bbb4c27 service cinder] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:41:44.621975 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf92c6d6-40f3-4bff-9190-c48e2bbb4c27 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:41:44.631124 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf92c6d6-40f3-4bff-9190-c48e2bbb4c27 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:41:44.632816 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf92c6d6-40f3-4bff-9190-c48e2bbb4c27 service cinder] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:41:44.644765 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1274/2529] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:41:44 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:41:44.675270 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce09a20-9b33-4502-a473-14a101ee7d2f None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:41:44.735478 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce09a20-9b33-4502-a473-14a101ee7d2f None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:41:44.740088 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d14df8-538c-4b1f-9188-b7b564acdf38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:41:44.740702 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d14df8-538c-4b1f-9188-b7b564acdf38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:41:44.740977 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d14df8-538c-4b1f-9188-b7b564acdf38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:41:44.741245 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d14df8-538c-4b1f-9188-b7b564acdf38 admin admin] PATH_INFO: `/v3/users/c94866c100a04a8791ca5eb5047f9c9d` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:41:44.742344 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d14df8-538c-4b1f-9188-b7b564acdf38 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c94866c100a04a8791ca5eb5047f9c9d)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:41:44.744253 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d14df8-538c-4b1f-9188-b7b564acdf38 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:41:44.789105 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-d7d14df8-538c-4b1f-9188-b7b564acdf38 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c94866c100a04a8791ca5eb5047f9c9d'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:41:44.790805 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:41:44.797206 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-d7d14df8-538c-4b1f-9188-b7b564acdf38 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c94866c100a04a8791ca5eb5047f9c9d'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:41:44.798769 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-d7d14df8-538c-4b1f-9188-b7b564acdf38 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c94866c100a04a8791ca5eb5047f9c9d'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:41:44.806040 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-d7d14df8-538c-4b1f-9188-b7b564acdf38 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c94866c100a04a8791ca5eb5047f9c9d'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:41:44.818334 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-d7d14df8-538c-4b1f-9188-b7b564acdf38 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c94866c100a04a8791ca5eb5047f9c9d'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:41:44.824224 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1256/2530] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Nov 2 07:41:44 2022] DELETE /identity/v3/users/c94866c100a04a8791ca5eb5047f9c9d => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 07:41:44.828693 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1362e9-0d20-49d8-8966-ddf9db35afc2 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:41:44.828973 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1362e9-0d20-49d8-8966-ddf9db35afc2 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:41:44.829244 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1362e9-0d20-49d8-8966-ddf9db35afc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:41:44.847650 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-7c1362e9-0d20-49d8-8966-ddf9db35afc2 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:41:44.848975 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-7c1362e9-0d20-49d8-8966-ddf9db35afc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:41:44.909134 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1275/2531] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:41:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:41:44.918788 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d14df8-538c-4b1f-9188-b7b564acdf38 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:41:44.920162 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-d7d14df8-538c-4b1f-9188-b7b564acdf38 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:41:44.980152 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d14df8-538c-4b1f-9188-b7b564acdf38 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:41:44.983863 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972222d5-1f25-4732-b31d-1ba1c8669f63 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3ec6ecb85f94ef59762877d9ea33ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:41:44.984457 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972222d5-1f25-4732-b31d-1ba1c8669f63 service neutron] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:41:44.984748 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972222d5-1f25-4732-b31d-1ba1c8669f63 service neutron] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:41:44.985007 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972222d5-1f25-4732-b31d-1ba1c8669f63 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:41:44.994598 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972222d5-1f25-4732-b31d-1ba1c8669f63 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:41:44.995568 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972222d5-1f25-4732-b31d-1ba1c8669f63 service neutron] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:41:45.005257 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1257/2532] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:41:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:41:45.280189 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1362e9-0d20-49d8-8966-ddf9db35afc2 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:41:45.281685 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-7c1362e9-0d20-49d8-8966-ddf9db35afc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:41:45.293579 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1362e9-0d20-49d8-8966-ddf9db35afc2 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:41:45.298795 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d86c10-7505-428c-adcb-0e136cb27530 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:41:45.299385 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d86c10-7505-428c-adcb-0e136cb27530 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:41:45.299665 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d86c10-7505-428c-adcb-0e136cb27530 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:41:45.299965 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d86c10-7505-428c-adcb-0e136cb27530 admin admin] PATH_INFO: `/v3/projects/f4b31ed293214e9d91670bef92f6b84a` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:41:45.301038 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d86c10-7505-428c-adcb-0e136cb27530 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f4b31ed293214e9d91670bef92f6b84a)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:41:45.302982 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d86c10-7505-428c-adcb-0e136cb27530 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:41:45.352073 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-69d86c10-7505-428c-adcb-0e136cb27530 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f4b31ed293214e9d91670bef92f6b84a', 'request_id': 'req-69d86c10-7505-428c-adcb-0e136cb27530'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:41:45.358680 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-69d86c10-7505-428c-adcb-0e136cb27530 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f4b31ed293214e9d91670bef92f6b84a', 'request_id': 'req-69d86c10-7505-428c-adcb-0e136cb27530'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:41:45.364994 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-69d86c10-7505-428c-adcb-0e136cb27530 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f4b31ed293214e9d91670bef92f6b84a', 'request_id': 'req-69d86c10-7505-428c-adcb-0e136cb27530'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:41:45.377217 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-69d86c10-7505-428c-adcb-0e136cb27530 admin admin] The token cache is being invalidate because project f4b31ed293214e9d91670bef92f6b84a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:41:45.377496 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-69d86c10-7505-428c-adcb-0e136cb27530 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:41:45.378103 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-69d86c10-7505-428c-adcb-0e136cb27530 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:41:45.379587 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1276/2533] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Nov 2 07:41:45 2022] DELETE /identity/v3/projects/f4b31ed293214e9d91670bef92f6b84a => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:41:45.383796 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972222d5-1f25-4732-b31d-1ba1c8669f63 service neutron] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:41:45.393145 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972222d5-1f25-4732-b31d-1ba1c8669f63 service neutron] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:41:45.448967 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f50415f-5448-4b69-bc89-20aa97f280ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:41:45.449586 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f50415f-5448-4b69-bc89-20aa97f280ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:41:45.449864 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f50415f-5448-4b69-bc89-20aa97f280ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:41:45.450132 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f50415f-5448-4b69-bc89-20aa97f280ab admin admin] PATH_INFO: `/v3/users/e4041b0193414bc08dce62e848ddf308` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:41:45.451254 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f50415f-5448-4b69-bc89-20aa97f280ab admin admin] RBAC: Authorizing `identity:delete_user(user_id=e4041b0193414bc08dce62e848ddf308)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:41:45.453143 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f50415f-5448-4b69-bc89-20aa97f280ab admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:41:45.500467 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-6f50415f-5448-4b69-bc89-20aa97f280ab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e4041b0193414bc08dce62e848ddf308'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:41:45.507933 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-6f50415f-5448-4b69-bc89-20aa97f280ab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e4041b0193414bc08dce62e848ddf308'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:41:45.508568 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-6f50415f-5448-4b69-bc89-20aa97f280ab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e4041b0193414bc08dce62e848ddf308'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:41:45.515555 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-6f50415f-5448-4b69-bc89-20aa97f280ab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e4041b0193414bc08dce62e848ddf308'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:41:45.521200 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:INSERT=2,keystone:DELETE=12,keystone:SELECT=86 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:41:45.530043 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-6f50415f-5448-4b69-bc89-20aa97f280ab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e4041b0193414bc08dce62e848ddf308'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:41:45.536632 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1258/2534] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Nov 2 07:41:45 2022] DELETE /identity/v3/users/e4041b0193414bc08dce62e848ddf308 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:41:45.801638 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d86c10-7505-428c-adcb-0e136cb27530 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:41:45.803156 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-69d86c10-7505-428c-adcb-0e136cb27530 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:41:45.816520 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d86c10-7505-428c-adcb-0e136cb27530 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:41:45.829072 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:41:45.878253 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401c9ffe-7459-4c25-9172-de3be205a3e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:41:45.878840 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401c9ffe-7459-4c25-9172-de3be205a3e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:41:45.879125 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401c9ffe-7459-4c25-9172-de3be205a3e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:41:45.879401 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401c9ffe-7459-4c25-9172-de3be205a3e8 admin admin] PATH_INFO: `/v3/projects/03232af795254812ab229b91b649176e` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:41:45.880536 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401c9ffe-7459-4c25-9172-de3be205a3e8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=03232af795254812ab229b91b649176e)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:41:45.882760 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401c9ffe-7459-4c25-9172-de3be205a3e8 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:41:45.928553 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-401c9ffe-7459-4c25-9172-de3be205a3e8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '03232af795254812ab229b91b649176e', 'request_id': 'req-401c9ffe-7459-4c25-9172-de3be205a3e8'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:41:45.934505 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-401c9ffe-7459-4c25-9172-de3be205a3e8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '03232af795254812ab229b91b649176e', 'request_id': 'req-401c9ffe-7459-4c25-9172-de3be205a3e8'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:41:45.940633 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-401c9ffe-7459-4c25-9172-de3be205a3e8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '03232af795254812ab229b91b649176e', 'request_id': 'req-401c9ffe-7459-4c25-9172-de3be205a3e8'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:41:45.952407 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-401c9ffe-7459-4c25-9172-de3be205a3e8 admin admin] The token cache is being invalidate because project 03232af795254812ab229b91b649176e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:41:45.952689 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-401c9ffe-7459-4c25-9172-de3be205a3e8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:41:45.953314 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-401c9ffe-7459-4c25-9172-de3be205a3e8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:41:45.954972 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1277/2535] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Nov 2 07:41:45 2022] DELETE /identity/v3/projects/03232af795254812ab229b91b649176e => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:41:45.960145 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5affa32b-67cd-4468-9d0f-afe06df7eaab None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:41:45.960428 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5affa32b-67cd-4468-9d0f-afe06df7eaab None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:41:45.960698 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5affa32b-67cd-4468-9d0f-afe06df7eaab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:41:45.980176 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-5affa32b-67cd-4468-9d0f-afe06df7eaab None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:41:45.981354 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-5affa32b-67cd-4468-9d0f-afe06df7eaab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:41:46.039243 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1259/2536] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:41:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:41:46.101032 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee5289b-3fec-42cb-8de2-b33786b2e969 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:41:46.101337 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee5289b-3fec-42cb-8de2-b33786b2e969 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:41:46.101611 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee5289b-3fec-42cb-8de2-b33786b2e969 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:41:46.121865 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-8ee5289b-3fec-42cb-8de2-b33786b2e969 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:41:46.123091 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-8ee5289b-3fec-42cb-8de2-b33786b2e969 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:41:46.131314 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1278/2537] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:41:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:41:46.135645 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5affa32b-67cd-4468-9d0f-afe06df7eaab None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:41:46.146047 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5affa32b-67cd-4468-9d0f-afe06df7eaab None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:41:46.150645 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70458337-2b72-4e7e-ad71-9e3f0d907a37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:41:46.151238 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70458337-2b72-4e7e-ad71-9e3f0d907a37 admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:41:46.151526 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70458337-2b72-4e7e-ad71-9e3f0d907a37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:41:46.151797 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70458337-2b72-4e7e-ad71-9e3f0d907a37 admin admin] PATH_INFO: `/v3/domains` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:41:46.152433 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70458337-2b72-4e7e-ad71-9e3f0d907a37 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:41:46.152736 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70458337-2b72-4e7e-ad71-9e3f0d907a37 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:41:46.154334 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70458337-2b72-4e7e-ad71-9e3f0d907a37 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:41:46.166191 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1260/2538] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Nov 2 07:41:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:41:46.170504 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee5289b-3fec-42cb-8de2-b33786b2e969 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:41:46.180228 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee5289b-3fec-42cb-8de2-b33786b2e969 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:41:46.184841 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35eb8f16-eada-4af3-a017-df9cf13d47cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:41:46.185430 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35eb8f16-eada-4af3-a017-df9cf13d47cc admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:41:46.185707 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35eb8f16-eada-4af3-a017-df9cf13d47cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:41:46.185975 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35eb8f16-eada-4af3-a017-df9cf13d47cc admin admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:41:46.186634 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35eb8f16-eada-4af3-a017-df9cf13d47cc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:41:46.188517 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35eb8f16-eada-4af3-a017-df9cf13d47cc admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:41:46.215240 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1279/2539] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Nov 2 07:41:46 2022] POST /identity/v3/projects => generated 383 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:41:46.220359 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70458337-2b72-4e7e-ad71-9e3f0d907a37 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:41:46.230245 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70458337-2b72-4e7e-ad71-9e3f0d907a37 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:41:46.289350 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa04dc2-ffec-4ecf-8955-ed3b434fe511 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:41:46.289946 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa04dc2-ffec-4ecf-8955-ed3b434fe511 admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:41:46.290228 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa04dc2-ffec-4ecf-8955-ed3b434fe511 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:41:46.290525 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa04dc2-ffec-4ecf-8955-ed3b434fe511 admin admin] PATH_INFO: `/v3/users` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:41:46.291135 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa04dc2-ffec-4ecf-8955-ed3b434fe511 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:41:46.293029 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa04dc2-ffec-4ecf-8955-ed3b434fe511 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:41:46.313971 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1261/2540] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Nov 2 07:41:46 2022] POST /identity/v3/users => generated 292 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 07:41:46.318035 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35eb8f16-eada-4af3-a017-df9cf13d47cc admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:41:46.328918 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35eb8f16-eada-4af3-a017-df9cf13d47cc admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:41:46.333641 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd65ad9a-fae7-4ef4-b43e-28d315051da4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:41:46.334238 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd65ad9a-fae7-4ef4-b43e-28d315051da4 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:41:46.334550 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd65ad9a-fae7-4ef4-b43e-28d315051da4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:41:46.334826 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd65ad9a-fae7-4ef4-b43e-28d315051da4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:41:46.335463 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd65ad9a-fae7-4ef4-b43e-28d315051da4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:41:46.337034 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd65ad9a-fae7-4ef4-b43e-28d315051da4 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:41:46.348239 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1280/2541] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:41:46 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:41:46.352791 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa04dc2-ffec-4ecf-8955-ed3b434fe511 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:41:46.364835 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa04dc2-ffec-4ecf-8955-ed3b434fe511 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:41:46.369530 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e655daf5-75dc-4782-a14e-30b56b6ccf08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:41:46.370146 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e655daf5-75dc-4782-a14e-30b56b6ccf08 admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:41:46.370452 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e655daf5-75dc-4782-a14e-30b56b6ccf08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:41:46.370723 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e655daf5-75dc-4782-a14e-30b56b6ccf08 admin admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:41:46.371355 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e655daf5-75dc-4782-a14e-30b56b6ccf08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:41:46.372977 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e655daf5-75dc-4782-a14e-30b56b6ccf08 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:41:46.387541 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1262/2542] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:41:46 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:41:46.391616 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd65ad9a-fae7-4ef4-b43e-28d315051da4 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:41:46.401678 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd65ad9a-fae7-4ef4-b43e-28d315051da4 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:41:46.406463 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd542d0-cd97-4e6a-b043-adf0a333db80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:41:46.407285 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd542d0-cd97-4e6a-b043-adf0a333db80 admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:41:46.407563 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd542d0-cd97-4e6a-b043-adf0a333db80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:41:46.407832 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd542d0-cd97-4e6a-b043-adf0a333db80 admin admin] PATH_INFO: `/v3/projects/5888171b32db49c8a3795b857c171d8b/users/71faf29054464f5c8a86f6effcf31320/roles/b494fedd00f24f0b84a529e8672a156a` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:41:46.423031 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd542d0-cd97-4e6a-b043-adf0a333db80 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5888171b32db49c8a3795b857c171d8b, user_id=71faf29054464f5c8a86f6effcf31320, role_id=b494fedd00f24f0b84a529e8672a156a)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:41:46.425639 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd542d0-cd97-4e6a-b043-adf0a333db80 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:41:46.438633 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1281/2543] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Nov 2 07:41:46 2022] PUT /identity/v3/projects/5888171b32db49c8a3795b857c171d8b/users/71faf29054464f5c8a86f6effcf31320/roles/b494fedd00f24f0b84a529e8672a156a => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 07:41:46.448431 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe94405-d429-4192-a2e8-9701c0486c4a None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:41:46.448715 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe94405-d429-4192-a2e8-9701c0486c4a None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:41:46.448993 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe94405-d429-4192-a2e8-9701c0486c4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:41:46.477081 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-fbe94405-d429-4192-a2e8-9701c0486c4a None None] MFA Rules not processed for user `71faf29054464f5c8a86f6effcf31320`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:41:46.478281 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-fbe94405-d429-4192-a2e8-9701c0486c4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:41:46.519174 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:41:46.542327 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1263/2544] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:41:46 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:41:46.546784 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd542d0-cd97-4e6a-b043-adf0a333db80 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:41:46.591572 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:41:46.610198 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd542d0-cd97-4e6a-b043-adf0a333db80 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:41:46.615163 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8080424-64ee-4729-86b6-69d92bfdde25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:41:46.615786 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8080424-64ee-4729-86b6-69d92bfdde25 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:41:46.616061 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8080424-64ee-4729-86b6-69d92bfdde25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:41:46.616376 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8080424-64ee-4729-86b6-69d92bfdde25 admin admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:41:46.616995 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8080424-64ee-4729-86b6-69d92bfdde25 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:41:46.619004 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8080424-64ee-4729-86b6-69d92bfdde25 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:41:46.647156 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1282/2545] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Nov 2 07:41:46 2022] POST /identity/v3/projects => generated 383 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:41:46.651016 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe94405-d429-4192-a2e8-9701c0486c4a None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:41:46.662367 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe94405-d429-4192-a2e8-9701c0486c4a None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:41:46.717151 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58683de9-0f20-4201-aca7-9f5d646172cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:41:46.717750 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58683de9-0f20-4201-aca7-9f5d646172cf admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:41:46.718032 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58683de9-0f20-4201-aca7-9f5d646172cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:41:46.718320 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58683de9-0f20-4201-aca7-9f5d646172cf admin admin] PATH_INFO: `/v3/users` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:41:46.719464 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58683de9-0f20-4201-aca7-9f5d646172cf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:41:46.721356 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58683de9-0f20-4201-aca7-9f5d646172cf admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:41:46.742943 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1264/2546] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Nov 2 07:41:46 2022] POST /identity/v3/users => generated 298 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:41:46.747285 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8080424-64ee-4729-86b6-69d92bfdde25 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:41:46.757729 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8080424-64ee-4729-86b6-69d92bfdde25 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:41:46.762288 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76aa6ccd-a8c4-4fa3-b36f-616fcb65078c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:41:46.762903 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76aa6ccd-a8c4-4fa3-b36f-616fcb65078c admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:41:46.763178 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76aa6ccd-a8c4-4fa3-b36f-616fcb65078c admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:41:46.763444 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76aa6ccd-a8c4-4fa3-b36f-616fcb65078c admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:41:46.764064 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76aa6ccd-a8c4-4fa3-b36f-616fcb65078c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:41:46.765637 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76aa6ccd-a8c4-4fa3-b36f-616fcb65078c admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:41:46.775930 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1283/2547] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:41:46 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:41:46.780155 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58683de9-0f20-4201-aca7-9f5d646172cf admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:41:46.790827 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58683de9-0f20-4201-aca7-9f5d646172cf admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:41:46.795384 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4d688f-ccbe-4e85-975f-736a32bf5753 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:41:46.796023 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4d688f-ccbe-4e85-975f-736a32bf5753 admin admin] REQUEST_METHOD: `PUT` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:41:46.796301 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4d688f-ccbe-4e85-975f-736a32bf5753 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:41:46.796571 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4d688f-ccbe-4e85-975f-736a32bf5753 admin admin] PATH_INFO: `/v3/projects/5f955d93f05e40a59fcc57730d942c0b/users/6450e3c33858414d9f32863c7e51ce5e/roles/c4bf019fc6c44c71a6c984f6d0bab75f` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:41:46.812581 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4d688f-ccbe-4e85-975f-736a32bf5753 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f955d93f05e40a59fcc57730d942c0b, user_id=6450e3c33858414d9f32863c7e51ce5e, role_id=c4bf019fc6c44c71a6c984f6d0bab75f)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:41:46.815298 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4d688f-ccbe-4e85-975f-736a32bf5753 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:41:46.827167 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1265/2548] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Nov 2 07:41:46 2022] PUT /identity/v3/projects/5f955d93f05e40a59fcc57730d942c0b/users/6450e3c33858414d9f32863c7e51ce5e/roles/c4bf019fc6c44c71a6c984f6d0bab75f => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:41:46.831379 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76aa6ccd-a8c4-4fa3-b36f-616fcb65078c admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:41:46.841198 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76aa6ccd-a8c4-4fa3-b36f-616fcb65078c admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:41:46.897340 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8333585-a537-4237-8553-33905a8868bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:41:46.897866 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8333585-a537-4237-8553-33905a8868bc admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:41:46.898143 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8333585-a537-4237-8553-33905a8868bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:41:46.898452 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8333585-a537-4237-8553-33905a8868bc admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:41:46.899079 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8333585-a537-4237-8553-33905a8868bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:41:46.900642 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8333585-a537-4237-8553-33905a8868bc admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:41:46.910913 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1284/2549] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:41:46 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:41:46.915174 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4d688f-ccbe-4e85-975f-736a32bf5753 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:41:46.925922 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4d688f-ccbe-4e85-975f-736a32bf5753 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:41:46.930550 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7634e268-c001-486d-96ca-44b97fc45f96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:41:46.931231 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7634e268-c001-486d-96ca-44b97fc45f96 admin admin] REQUEST_METHOD: `PUT` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:41:46.931512 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7634e268-c001-486d-96ca-44b97fc45f96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:41:46.931786 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7634e268-c001-486d-96ca-44b97fc45f96 admin admin] PATH_INFO: `/v3/projects/5f955d93f05e40a59fcc57730d942c0b/users/6450e3c33858414d9f32863c7e51ce5e/roles/c4bf019fc6c44c71a6c984f6d0bab75f` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:41:46.933689 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7634e268-c001-486d-96ca-44b97fc45f96 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f955d93f05e40a59fcc57730d942c0b, user_id=6450e3c33858414d9f32863c7e51ce5e, role_id=c4bf019fc6c44c71a6c984f6d0bab75f)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:41:46.936332 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7634e268-c001-486d-96ca-44b97fc45f96 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:41:46.947213 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1266/2550] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Nov 2 07:41:46 2022] PUT /identity/v3/projects/5f955d93f05e40a59fcc57730d942c0b/users/6450e3c33858414d9f32863c7e51ce5e/roles/c4bf019fc6c44c71a6c984f6d0bab75f => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:41:46.957836 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a10cf40-cbb3-4292-9ec0-9408c2480337 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:41:46.958143 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a10cf40-cbb3-4292-9ec0-9408c2480337 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:41:46.958436 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a10cf40-cbb3-4292-9ec0-9408c2480337 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:41:46.986484 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-6a10cf40-cbb3-4292-9ec0-9408c2480337 None None] MFA Rules not processed for user `6450e3c33858414d9f32863c7e51ce5e`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:41:46.987688 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-6a10cf40-cbb3-4292-9ec0-9408c2480337 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:41:47.056430 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1285/2551] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:41:46 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:41:47.068524 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7634e268-c001-486d-96ca-44b97fc45f96 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:41:47.070203 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-7634e268-c001-486d-96ca-44b97fc45f96 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:41:47.126249 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7634e268-c001-486d-96ca-44b97fc45f96 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:41:47.129802 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94136f8-a160-42d2-a95d-bbfef4ab0cc2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43bb326fb82e4d2c83fb2eaf91860dc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:41:47.130407 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94136f8-a160-42d2-a95d-bbfef4ab0cc2 service cinder] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:41:47.130687 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94136f8-a160-42d2-a95d-bbfef4ab0cc2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:41:47.130953 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94136f8-a160-42d2-a95d-bbfef4ab0cc2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:41:47.139879 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94136f8-a160-42d2-a95d-bbfef4ab0cc2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:41:47.141839 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94136f8-a160-42d2-a95d-bbfef4ab0cc2 service cinder] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:41:47.151045 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1267/2552] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:41:47 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:41:47.226775 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a10cf40-cbb3-4292-9ec0-9408c2480337 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:41:47.237323 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a10cf40-cbb3-4292-9ec0-9408c2480337 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:41:47.240996 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76033287-b31a-4967-aab8-f2740d06e21a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43bb326fb82e4d2c83fb2eaf91860dc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:41:47.241618 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76033287-b31a-4967-aab8-f2740d06e21a service cinder] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:41:47.241889 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76033287-b31a-4967-aab8-f2740d06e21a service cinder] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:41:47.242156 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76033287-b31a-4967-aab8-f2740d06e21a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:41:47.276953 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:41:47.302060 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76033287-b31a-4967-aab8-f2740d06e21a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:41:47.303508 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76033287-b31a-4967-aab8-f2740d06e21a service cinder] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:41:47.312275 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1286/2553] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:41:47 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:41:47.578661 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94136f8-a160-42d2-a95d-bbfef4ab0cc2 service cinder] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:41:47.619717 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:41:47.641448 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94136f8-a160-42d2-a95d-bbfef4ab0cc2 service cinder] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:41:47.645987 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71928a37-ed4a-4a1c-8783-fbe1521c5ab7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:41:47.647055 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71928a37-ed4a-4a1c-8783-fbe1521c5ab7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:41:47.647334 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71928a37-ed4a-4a1c-8783-fbe1521c5ab7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:41:47.647603 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71928a37-ed4a-4a1c-8783-fbe1521c5ab7 admin admin] PATH_INFO: `/v3/users/71faf29054464f5c8a86f6effcf31320` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:41:47.648751 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71928a37-ed4a-4a1c-8783-fbe1521c5ab7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=71faf29054464f5c8a86f6effcf31320)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:41:47.650687 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71928a37-ed4a-4a1c-8783-fbe1521c5ab7 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:41:47.696687 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-71928a37-ed4a-4a1c-8783-fbe1521c5ab7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '71faf29054464f5c8a86f6effcf31320'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:41:47.704653 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-71928a37-ed4a-4a1c-8783-fbe1521c5ab7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '71faf29054464f5c8a86f6effcf31320'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:41:47.705313 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-71928a37-ed4a-4a1c-8783-fbe1521c5ab7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '71faf29054464f5c8a86f6effcf31320'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:41:47.712388 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-71928a37-ed4a-4a1c-8783-fbe1521c5ab7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '71faf29054464f5c8a86f6effcf31320'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:41:47.724493 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-71928a37-ed4a-4a1c-8783-fbe1521c5ab7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '71faf29054464f5c8a86f6effcf31320'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:41:47.730430 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1268/2554] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Nov 2 07:41:47 2022] DELETE /identity/v3/users/71faf29054464f5c8a86f6effcf31320 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:41:47.735657 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8ef8d2-5c6f-4a0d-9e76-df98a0420438 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:41:47.735936 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8ef8d2-5c6f-4a0d-9e76-df98a0420438 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:41:47.736208 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8ef8d2-5c6f-4a0d-9e76-df98a0420438 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:41:47.754754 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-4d8ef8d2-5c6f-4a0d-9e76-df98a0420438 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:41:47.755936 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-4d8ef8d2-5c6f-4a0d-9e76-df98a0420438 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:41:47.813497 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1287/2555] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:41:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:41:47.823262 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71928a37-ed4a-4a1c-8783-fbe1521c5ab7 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:41:47.824695 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-71928a37-ed4a-4a1c-8783-fbe1521c5ab7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:41:47.883236 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71928a37-ed4a-4a1c-8783-fbe1521c5ab7 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:41:47.886887 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cede8b1-0b07-4d57-ac35-c16abe07c7a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3ec6ecb85f94ef59762877d9ea33ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:41:47.887481 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cede8b1-0b07-4d57-ac35-c16abe07c7a9 service neutron] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:41:47.887756 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cede8b1-0b07-4d57-ac35-c16abe07c7a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:41:47.888022 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cede8b1-0b07-4d57-ac35-c16abe07c7a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:41:47.897360 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cede8b1-0b07-4d57-ac35-c16abe07c7a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:41:47.898801 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cede8b1-0b07-4d57-ac35-c16abe07c7a9 service neutron] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:41:47.908590 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1269/2556] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:41:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 07:41:48.179061 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8ef8d2-5c6f-4a0d-9e76-df98a0420438 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:41:48.180506 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-4d8ef8d2-5c6f-4a0d-9e76-df98a0420438 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:41:48.192266 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8ef8d2-5c6f-4a0d-9e76-df98a0420438 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:41:48.196823 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946b6f63-3631-48ad-8ba4-70c7630d0dd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:41:48.197456 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946b6f63-3631-48ad-8ba4-70c7630d0dd9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:41:48.197733 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946b6f63-3631-48ad-8ba4-70c7630d0dd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:41:48.198001 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946b6f63-3631-48ad-8ba4-70c7630d0dd9 admin admin] PATH_INFO: `/v3/projects/5888171b32db49c8a3795b857c171d8b` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:41:48.199562 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946b6f63-3631-48ad-8ba4-70c7630d0dd9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5888171b32db49c8a3795b857c171d8b)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:41:48.201496 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946b6f63-3631-48ad-8ba4-70c7630d0dd9 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:41:48.254570 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-946b6f63-3631-48ad-8ba4-70c7630d0dd9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5888171b32db49c8a3795b857c171d8b', 'request_id': 'req-946b6f63-3631-48ad-8ba4-70c7630d0dd9'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:41:48.261447 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-946b6f63-3631-48ad-8ba4-70c7630d0dd9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5888171b32db49c8a3795b857c171d8b', 'request_id': 'req-946b6f63-3631-48ad-8ba4-70c7630d0dd9'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:41:48.267756 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-946b6f63-3631-48ad-8ba4-70c7630d0dd9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5888171b32db49c8a3795b857c171d8b', 'request_id': 'req-946b6f63-3631-48ad-8ba4-70c7630d0dd9'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:41:48.279585 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-946b6f63-3631-48ad-8ba4-70c7630d0dd9 admin admin] The token cache is being invalidate because project 5888171b32db49c8a3795b857c171d8b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:41:48.279900 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-946b6f63-3631-48ad-8ba4-70c7630d0dd9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:41:48.280569 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-946b6f63-3631-48ad-8ba4-70c7630d0dd9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:41:48.282865 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1288/2557] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Nov 2 07:41:48 2022] DELETE /identity/v3/projects/5888171b32db49c8a3795b857c171d8b => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:41:48.286870 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cede8b1-0b07-4d57-ac35-c16abe07c7a9 service neutron] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:41:48.296283 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cede8b1-0b07-4d57-ac35-c16abe07c7a9 service neutron] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:41:48.352025 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8727836-dd6f-4422-aa79-1044bcf8f0af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:41:48.352647 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8727836-dd6f-4422-aa79-1044bcf8f0af admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:41:48.352927 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8727836-dd6f-4422-aa79-1044bcf8f0af admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:41:48.353199 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8727836-dd6f-4422-aa79-1044bcf8f0af admin admin] PATH_INFO: `/v3/users/6450e3c33858414d9f32863c7e51ce5e` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:41:48.354300 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8727836-dd6f-4422-aa79-1044bcf8f0af admin admin] RBAC: Authorizing `identity:delete_user(user_id=6450e3c33858414d9f32863c7e51ce5e)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:41:48.356234 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8727836-dd6f-4422-aa79-1044bcf8f0af admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:41:48.357640 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:41:48.405492 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-b8727836-dd6f-4422-aa79-1044bcf8f0af admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6450e3c33858414d9f32863c7e51ce5e'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:41:48.414636 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-b8727836-dd6f-4422-aa79-1044bcf8f0af admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6450e3c33858414d9f32863c7e51ce5e'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:41:48.415253 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-b8727836-dd6f-4422-aa79-1044bcf8f0af admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6450e3c33858414d9f32863c7e51ce5e'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:41:48.422839 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-b8727836-dd6f-4422-aa79-1044bcf8f0af admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6450e3c33858414d9f32863c7e51ce5e'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:41:48.434498 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-b8727836-dd6f-4422-aa79-1044bcf8f0af admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6450e3c33858414d9f32863c7e51ce5e'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:41:48.440464 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1270/2558] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Nov 2 07:41:48 2022] DELETE /identity/v3/users/6450e3c33858414d9f32863c7e51ce5e => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 07:41:48.714770 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946b6f63-3631-48ad-8ba4-70c7630d0dd9 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:41:48.716230 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-946b6f63-3631-48ad-8ba4-70c7630d0dd9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:41:48.730290 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946b6f63-3631-48ad-8ba4-70c7630d0dd9 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:41:48.789124 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1993af-99b9-41e8-8471-a5e3dafe0577 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:41:48.789681 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1993af-99b9-41e8-8471-a5e3dafe0577 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:41:48.789955 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1993af-99b9-41e8-8471-a5e3dafe0577 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:41:48.790224 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1993af-99b9-41e8-8471-a5e3dafe0577 admin admin] PATH_INFO: `/v3/projects/5f955d93f05e40a59fcc57730d942c0b` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:41:48.791412 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1993af-99b9-41e8-8471-a5e3dafe0577 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5f955d93f05e40a59fcc57730d942c0b)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:41:48.793364 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1993af-99b9-41e8-8471-a5e3dafe0577 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:41:48.794767 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:41:48.841076 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-9d1993af-99b9-41e8-8471-a5e3dafe0577 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5f955d93f05e40a59fcc57730d942c0b', 'request_id': 'req-9d1993af-99b9-41e8-8471-a5e3dafe0577'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:41:48.847145 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-9d1993af-99b9-41e8-8471-a5e3dafe0577 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5f955d93f05e40a59fcc57730d942c0b', 'request_id': 'req-9d1993af-99b9-41e8-8471-a5e3dafe0577'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:41:48.852760 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-9d1993af-99b9-41e8-8471-a5e3dafe0577 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5f955d93f05e40a59fcc57730d942c0b', 'request_id': 'req-9d1993af-99b9-41e8-8471-a5e3dafe0577'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:41:48.865500 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-9d1993af-99b9-41e8-8471-a5e3dafe0577 admin admin] The token cache is being invalidate because project 5f955d93f05e40a59fcc57730d942c0b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:41:48.865785 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-9d1993af-99b9-41e8-8471-a5e3dafe0577 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:41:48.866435 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-9d1993af-99b9-41e8-8471-a5e3dafe0577 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:41:48.867946 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1289/2559] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Nov 2 07:41:48 2022] DELETE /identity/v3/projects/5f955d93f05e40a59fcc57730d942c0b => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:41:48.872829 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e479a3c-4134-4b1b-9cf9-62aa520de0d7 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:41:48.873302 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e479a3c-4134-4b1b-9cf9-62aa520de0d7 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:41:48.873421 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e479a3c-4134-4b1b-9cf9-62aa520de0d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:41:48.893274 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-3e479a3c-4134-4b1b-9cf9-62aa520de0d7 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:41:48.894528 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-3e479a3c-4134-4b1b-9cf9-62aa520de0d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:41:48.952496 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1271/2560] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:41:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:41:48.960122 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9b8f4e-29aa-464e-8718-4e14bc2a6284 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:41:48.960415 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9b8f4e-29aa-464e-8718-4e14bc2a6284 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:41:48.960695 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9b8f4e-29aa-464e-8718-4e14bc2a6284 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:41:48.981621 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-4c9b8f4e-29aa-464e-8718-4e14bc2a6284 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:41:48.982842 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-4c9b8f4e-29aa-464e-8718-4e14bc2a6284 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:41:48.992600 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1290/2561] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:41:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:41:48.997250 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e479a3c-4134-4b1b-9cf9-62aa520de0d7 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:41:49.007631 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e479a3c-4134-4b1b-9cf9-62aa520de0d7 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:41:49.012200 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f051451a-eb8b-4712-9d96-f3b8c9982ff7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:41:49.012790 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f051451a-eb8b-4712-9d96-f3b8c9982ff7 admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:41:49.013073 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f051451a-eb8b-4712-9d96-f3b8c9982ff7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:41:49.013344 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f051451a-eb8b-4712-9d96-f3b8c9982ff7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:41:49.013981 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f051451a-eb8b-4712-9d96-f3b8c9982ff7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:41:49.014285 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f051451a-eb8b-4712-9d96-f3b8c9982ff7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:41:49.016225 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f051451a-eb8b-4712-9d96-f3b8c9982ff7 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:41:49.027817 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1272/2562] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Nov 2 07:41:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:41:49.032572 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9b8f4e-29aa-464e-8718-4e14bc2a6284 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:41:49.041666 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9b8f4e-29aa-464e-8718-4e14bc2a6284 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:41:49.046142 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605d7ac2-5c4c-402f-9300-d88e002be442 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:41:49.047352 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605d7ac2-5c4c-402f-9300-d88e002be442 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:41:49.047712 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605d7ac2-5c4c-402f-9300-d88e002be442 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:41:49.047982 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605d7ac2-5c4c-402f-9300-d88e002be442 admin admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:41:49.048608 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605d7ac2-5c4c-402f-9300-d88e002be442 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:41:49.050498 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605d7ac2-5c4c-402f-9300-d88e002be442 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:41:49.077951 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1291/2563] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Nov 2 07:41:49 2022] POST /identity/v3/projects => generated 373 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:41:49.082034 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f051451a-eb8b-4712-9d96-f3b8c9982ff7 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:41:49.091890 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f051451a-eb8b-4712-9d96-f3b8c9982ff7 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:41:49.147324 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c34a5fc-1ec6-4c36-af54-c472ceb8a751 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:41:49.147917 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c34a5fc-1ec6-4c36-af54-c472ceb8a751 admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:41:49.148193 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c34a5fc-1ec6-4c36-af54-c472ceb8a751 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:41:49.148459 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c34a5fc-1ec6-4c36-af54-c472ceb8a751 admin admin] PATH_INFO: `/v3/users` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:41:49.149061 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c34a5fc-1ec6-4c36-af54-c472ceb8a751 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:41:49.150989 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c34a5fc-1ec6-4c36-af54-c472ceb8a751 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:41:49.162260 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:41:49.174748 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1273/2564] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Nov 2 07:41:49 2022] POST /identity/v3/users => generated 287 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:41:49.179041 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605d7ac2-5c4c-402f-9300-d88e002be442 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:41:49.189594 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605d7ac2-5c4c-402f-9300-d88e002be442 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:41:49.195055 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbb1980-8ac2-4b6f-80ae-3a1c3aa75b51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:41:49.195685 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbb1980-8ac2-4b6f-80ae-3a1c3aa75b51 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:41:49.195958 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbb1980-8ac2-4b6f-80ae-3a1c3aa75b51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:41:49.196221 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbb1980-8ac2-4b6f-80ae-3a1c3aa75b51 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:41:49.196845 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbb1980-8ac2-4b6f-80ae-3a1c3aa75b51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:41:49.198556 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbb1980-8ac2-4b6f-80ae-3a1c3aa75b51 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:41:49.208667 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1292/2565] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:41:49 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:41:49.212959 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c34a5fc-1ec6-4c36-af54-c472ceb8a751 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:41:49.224633 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c34a5fc-1ec6-4c36-af54-c472ceb8a751 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:41:49.229165 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923ca10c-6341-4164-aaed-db741fe57f7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:41:49.229748 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923ca10c-6341-4164-aaed-db741fe57f7f admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:41:49.230024 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923ca10c-6341-4164-aaed-db741fe57f7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:41:49.230293 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923ca10c-6341-4164-aaed-db741fe57f7f admin admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:41:49.230945 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923ca10c-6341-4164-aaed-db741fe57f7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:41:49.232526 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923ca10c-6341-4164-aaed-db741fe57f7f admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:41:49.243840 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1274/2566] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:41:49 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:41:49.248234 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbb1980-8ac2-4b6f-80ae-3a1c3aa75b51 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:41:49.259070 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbb1980-8ac2-4b6f-80ae-3a1c3aa75b51 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:41:49.263709 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad15d2b9-c779-4274-88b9-32b29e7aff98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:41:49.264352 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad15d2b9-c779-4274-88b9-32b29e7aff98 admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:41:49.264632 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad15d2b9-c779-4274-88b9-32b29e7aff98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:41:49.264906 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad15d2b9-c779-4274-88b9-32b29e7aff98 admin admin] PATH_INFO: `/v3/projects/90b73ff5e6e44a409ce3a0220a0c3980/users/6cf30037ab094c3e9b5ab4b73197b99b/roles/b494fedd00f24f0b84a529e8672a156a` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:41:49.279924 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad15d2b9-c779-4274-88b9-32b29e7aff98 admin admin] RBAC: Authorizing `identity:create_grant(project_id=90b73ff5e6e44a409ce3a0220a0c3980, user_id=6cf30037ab094c3e9b5ab4b73197b99b, role_id=b494fedd00f24f0b84a529e8672a156a)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:41:49.282626 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad15d2b9-c779-4274-88b9-32b29e7aff98 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:41:49.295161 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1293/2567] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Nov 2 07:41:49 2022] PUT /identity/v3/projects/90b73ff5e6e44a409ce3a0220a0c3980/users/6cf30037ab094c3e9b5ab4b73197b99b/roles/b494fedd00f24f0b84a529e8672a156a => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:41:49.300774 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6421eba3-ebe4-4c6c-bcf7-3892a29bc853 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:41:49.301075 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6421eba3-ebe4-4c6c-bcf7-3892a29bc853 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:41:49.301336 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6421eba3-ebe4-4c6c-bcf7-3892a29bc853 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:41:49.320624 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-6421eba3-ebe4-4c6c-bcf7-3892a29bc853 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:41:49.321812 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-6421eba3-ebe4-4c6c-bcf7-3892a29bc853 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:41:49.381088 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1275/2568] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:41:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:41:49.391929 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad15d2b9-c779-4274-88b9-32b29e7aff98 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:41:49.393575 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-ad15d2b9-c779-4274-88b9-32b29e7aff98 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:41:49.449540 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad15d2b9-c779-4274-88b9-32b29e7aff98 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:41:49.453627 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6e6667-c84a-4180-9c5c-8e622329a832 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3ec6ecb85f94ef59762877d9ea33ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:41:49.454277 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6e6667-c84a-4180-9c5c-8e622329a832 service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:41:49.454581 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6e6667-c84a-4180-9c5c-8e622329a832 service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:41:49.454848 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6e6667-c84a-4180-9c5c-8e622329a832 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:41:49.463651 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6e6667-c84a-4180-9c5c-8e622329a832 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:41:49.465184 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6e6667-c84a-4180-9c5c-8e622329a832 service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:41:49.474459 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1294/2569] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:41:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:41:50.066088 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b857ef3e-ce04-4f60-a5f3-f1bc26619376 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:41:50.066669 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b857ef3e-ce04-4f60-a5f3-f1bc26619376 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:41:50.066669 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b857ef3e-ce04-4f60-a5f3-f1bc26619376 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:41:50.094916 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-b857ef3e-ce04-4f60-a5f3-f1bc26619376 None None] MFA Rules not processed for user `6cf30037ab094c3e9b5ab4b73197b99b`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:41:50.096135 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-b857ef3e-ce04-4f60-a5f3-f1bc26619376 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:41:50.161347 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1276/2570] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:41:50 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:41:50.165815 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6e6667-c84a-4180-9c5c-8e622329a832 service neutron] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:41:50.175758 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6e6667-c84a-4180-9c5c-8e622329a832 service neutron] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:41:50.180258 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eeddcf4-e4cc-4e20-b53d-1c76f4feb92c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:41:50.180853 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eeddcf4-e4cc-4e20-b53d-1c76f4feb92c admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:41:50.181127 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eeddcf4-e4cc-4e20-b53d-1c76f4feb92c admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:41:50.181392 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eeddcf4-e4cc-4e20-b53d-1c76f4feb92c admin admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:41:50.182010 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eeddcf4-e4cc-4e20-b53d-1c76f4feb92c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:41:50.184051 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eeddcf4-e4cc-4e20-b53d-1c76f4feb92c admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:41:50.211821 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1295/2571] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Nov 2 07:41:50 2022] POST /identity/v3/projects => generated 373 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:41:50.215847 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b857ef3e-ce04-4f60-a5f3-f1bc26619376 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:41:50.226677 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b857ef3e-ce04-4f60-a5f3-f1bc26619376 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:41:50.288893 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f9f6ea-f909-40d3-90c4-383e8bbb99f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:41:50.289495 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f9f6ea-f909-40d3-90c4-383e8bbb99f9 admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:41:50.289768 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f9f6ea-f909-40d3-90c4-383e8bbb99f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:41:50.290033 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f9f6ea-f909-40d3-90c4-383e8bbb99f9 admin admin] PATH_INFO: `/v3/users` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:41:50.290686 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f9f6ea-f909-40d3-90c4-383e8bbb99f9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:41:50.292576 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f9f6ea-f909-40d3-90c4-383e8bbb99f9 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:41:50.313339 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1277/2572] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Nov 2 07:41:50 2022] POST /identity/v3/users => generated 293 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:41:50.317428 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eeddcf4-e4cc-4e20-b53d-1c76f4feb92c admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:41:50.328295 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eeddcf4-e4cc-4e20-b53d-1c76f4feb92c admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:41:50.332830 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf21d5f-3a72-4608-acba-34670a39a99a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:41:50.333417 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf21d5f-3a72-4608-acba-34670a39a99a admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:41:50.333692 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf21d5f-3a72-4608-acba-34670a39a99a admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:41:50.333959 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf21d5f-3a72-4608-acba-34670a39a99a admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:41:50.334650 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf21d5f-3a72-4608-acba-34670a39a99a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:41:50.336268 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf21d5f-3a72-4608-acba-34670a39a99a admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:41:50.337465 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:41:50.351590 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1296/2573] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:41:50 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:41:50.352825 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f9f6ea-f909-40d3-90c4-383e8bbb99f9 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:41:50.359464 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:41:50.363548 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f9f6ea-f909-40d3-90c4-383e8bbb99f9 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:41:50.369271 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0341e37-e656-4111-937b-15d1a973c377 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:41:50.369938 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0341e37-e656-4111-937b-15d1a973c377 admin admin] REQUEST_METHOD: `PUT` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:41:50.370218 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0341e37-e656-4111-937b-15d1a973c377 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:41:50.370529 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0341e37-e656-4111-937b-15d1a973c377 admin admin] PATH_INFO: `/v3/projects/1c7a918ce15c4294b1a5c9cf52773791/users/2981733989e64de1839966e78bbcb014/roles/c4bf019fc6c44c71a6c984f6d0bab75f` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:41:50.385856 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0341e37-e656-4111-937b-15d1a973c377 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c7a918ce15c4294b1a5c9cf52773791, user_id=2981733989e64de1839966e78bbcb014, role_id=c4bf019fc6c44c71a6c984f6d0bab75f)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:41:50.388511 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0341e37-e656-4111-937b-15d1a973c377 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:41:50.401818 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1278/2574] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Nov 2 07:41:50 2022] PUT /identity/v3/projects/1c7a918ce15c4294b1a5c9cf52773791/users/2981733989e64de1839966e78bbcb014/roles/c4bf019fc6c44c71a6c984f6d0bab75f => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:41:50.405840 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf21d5f-3a72-4608-acba-34670a39a99a admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:41:50.415598 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf21d5f-3a72-4608-acba-34670a39a99a admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:41:50.474075 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad1496d-3c43-492b-a5de-2654f63f03c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:41:50.474709 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad1496d-3c43-492b-a5de-2654f63f03c2 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:41:50.474990 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad1496d-3c43-492b-a5de-2654f63f03c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:41:50.475260 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad1496d-3c43-492b-a5de-2654f63f03c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:41:50.475883 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad1496d-3c43-492b-a5de-2654f63f03c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:41:50.477443 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad1496d-3c43-492b-a5de-2654f63f03c2 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:41:50.486823 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1297/2575] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:41:50 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:41:50.491221 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0341e37-e656-4111-937b-15d1a973c377 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:41:50.502692 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0341e37-e656-4111-937b-15d1a973c377 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:41:50.507535 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ab1a6a-22f6-4252-942a-1a7805502058 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:41:50.508170 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ab1a6a-22f6-4252-942a-1a7805502058 admin admin] REQUEST_METHOD: `PUT` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:41:50.508447 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ab1a6a-22f6-4252-942a-1a7805502058 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:41:50.508742 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ab1a6a-22f6-4252-942a-1a7805502058 admin admin] PATH_INFO: `/v3/projects/1c7a918ce15c4294b1a5c9cf52773791/users/2981733989e64de1839966e78bbcb014/roles/c4bf019fc6c44c71a6c984f6d0bab75f` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:41:50.510641 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ab1a6a-22f6-4252-942a-1a7805502058 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c7a918ce15c4294b1a5c9cf52773791, user_id=2981733989e64de1839966e78bbcb014, role_id=c4bf019fc6c44c71a6c984f6d0bab75f)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:41:50.513395 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ab1a6a-22f6-4252-942a-1a7805502058 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:41:50.524439 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1279/2576] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Nov 2 07:41:50 2022] PUT /identity/v3/projects/1c7a918ce15c4294b1a5c9cf52773791/users/2981733989e64de1839966e78bbcb014/roles/c4bf019fc6c44c71a6c984f6d0bab75f => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:41:51.148869 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e07f36c-4c09-49f2-bece-ef0dc076e0dd None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:41:51.149223 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e07f36c-4c09-49f2-bece-ef0dc076e0dd None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:41:51.149454 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e07f36c-4c09-49f2-bece-ef0dc076e0dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:41:51.178212 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-1e07f36c-4c09-49f2-bece-ef0dc076e0dd None None] MFA Rules not processed for user `2981733989e64de1839966e78bbcb014`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:41:51.179626 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-1e07f36c-4c09-49f2-bece-ef0dc076e0dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:41:51.249657 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1298/2577] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:41:51 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:41:51.261567 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ab1a6a-22f6-4252-942a-1a7805502058 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:41:51.263310 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-e0ab1a6a-22f6-4252-942a-1a7805502058 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:41:51.322261 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ab1a6a-22f6-4252-942a-1a7805502058 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:41:51.325862 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8890b500-6398-4e05-8a8d-296b91a1ba68 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43bb326fb82e4d2c83fb2eaf91860dc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:41:51.326502 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8890b500-6398-4e05-8a8d-296b91a1ba68 service cinder] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:41:51.326785 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8890b500-6398-4e05-8a8d-296b91a1ba68 service cinder] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:41:51.327058 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8890b500-6398-4e05-8a8d-296b91a1ba68 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:41:51.387884 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8890b500-6398-4e05-8a8d-296b91a1ba68 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:41:51.389312 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8890b500-6398-4e05-8a8d-296b91a1ba68 service cinder] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:41:51.399075 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1280/2578] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:41:51 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:41:52.765770 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e07f36c-4c09-49f2-bece-ef0dc076e0dd None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:41:52.767187 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-1e07f36c-4c09-49f2-bece-ef0dc076e0dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:41:52.778758 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e07f36c-4c09-49f2-bece-ef0dc076e0dd None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:41:52.782304 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af9774c-7904-4842-85c4-cbfed7b7025c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43bb326fb82e4d2c83fb2eaf91860dc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:41:52.782925 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af9774c-7904-4842-85c4-cbfed7b7025c service cinder] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:41:52.783198 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af9774c-7904-4842-85c4-cbfed7b7025c service cinder] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:41:52.783469 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af9774c-7904-4842-85c4-cbfed7b7025c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:41:52.792337 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af9774c-7904-4842-85c4-cbfed7b7025c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:41:52.793736 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af9774c-7904-4842-85c4-cbfed7b7025c service cinder] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:41:52.803095 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1299/2579] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:41:52 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:42:01.395115 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=61,keystone:INSERT=2 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:42:02.799006 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=71 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:42:05.472104 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8890b500-6398-4e05-8a8d-296b91a1ba68 service cinder] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:42:05.473634 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-8890b500-6398-4e05-8a8d-296b91a1ba68 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:42:05.550124 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8890b500-6398-4e05-8a8d-296b91a1ba68 service cinder] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:42:05.555162 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa061015-e292-409d-8203-15a24827df2e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2fc6a156c54d0c9d325901d0f7f6fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:42:05.555699 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa061015-e292-409d-8203-15a24827df2e service placement] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:42:05.555978 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa061015-e292-409d-8203-15a24827df2e service placement] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:42:05.556246 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa061015-e292-409d-8203-15a24827df2e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:42:05.557945 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-aa061015-e292-409d-8203-15a24827df2e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:42:05.569854 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa061015-e292-409d-8203-15a24827df2e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:42:05.571666 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa061015-e292-409d-8203-15a24827df2e service placement] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:42:05.582172 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1281/2580] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:42:05 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:42:06.055176 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af9774c-7904-4842-85c4-cbfed7b7025c service cinder] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:42:06.116033 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af9774c-7904-4842-85c4-cbfed7b7025c service cinder] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:42:06.120382 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db1743a-6b19-4794-bf54-7942b2544bbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:42:06.120991 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db1743a-6b19-4794-bf54-7942b2544bbf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:42:06.121266 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db1743a-6b19-4794-bf54-7942b2544bbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:42:06.121533 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db1743a-6b19-4794-bf54-7942b2544bbf admin admin] PATH_INFO: `/v3/users/6cf30037ab094c3e9b5ab4b73197b99b` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:42:06.123279 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db1743a-6b19-4794-bf54-7942b2544bbf admin admin] RBAC: Authorizing `identity:delete_user(user_id=6cf30037ab094c3e9b5ab4b73197b99b)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:42:06.125214 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db1743a-6b19-4794-bf54-7942b2544bbf admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:42:06.169407 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-3db1743a-6b19-4794-bf54-7942b2544bbf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6cf30037ab094c3e9b5ab4b73197b99b'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:42:06.175847 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-3db1743a-6b19-4794-bf54-7942b2544bbf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6cf30037ab094c3e9b5ab4b73197b99b'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:42:06.176479 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-3db1743a-6b19-4794-bf54-7942b2544bbf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6cf30037ab094c3e9b5ab4b73197b99b'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:42:06.183237 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-3db1743a-6b19-4794-bf54-7942b2544bbf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6cf30037ab094c3e9b5ab4b73197b99b'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:42:06.183854 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-3db1743a-6b19-4794-bf54-7942b2544bbf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6cf30037ab094c3e9b5ab4b73197b99b'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:42:06.201182 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1300/2581] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Nov 2 07:42:06 2022] DELETE /identity/v3/users/6cf30037ab094c3e9b5ab4b73197b99b => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:42:06.380593 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa061015-e292-409d-8203-15a24827df2e service placement] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:42:06.392491 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa061015-e292-409d-8203-15a24827df2e service placement] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:42:06.450255 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697c2fb8-2756-4265-9d88-e43c00d6eb5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:42:06.451099 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697c2fb8-2756-4265-9d88-e43c00d6eb5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:42:06.451579 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697c2fb8-2756-4265-9d88-e43c00d6eb5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:42:06.452051 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697c2fb8-2756-4265-9d88-e43c00d6eb5b admin admin] PATH_INFO: `/v3/projects/90b73ff5e6e44a409ce3a0220a0c3980` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:42:06.453672 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697c2fb8-2756-4265-9d88-e43c00d6eb5b admin admin] RBAC: Authorizing `identity:delete_project(project_id=90b73ff5e6e44a409ce3a0220a0c3980)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:42:06.455920 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697c2fb8-2756-4265-9d88-e43c00d6eb5b admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:42:06.503436 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-697c2fb8-2756-4265-9d88-e43c00d6eb5b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '90b73ff5e6e44a409ce3a0220a0c3980', 'request_id': 'req-697c2fb8-2756-4265-9d88-e43c00d6eb5b'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:42:06.509551 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-697c2fb8-2756-4265-9d88-e43c00d6eb5b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '90b73ff5e6e44a409ce3a0220a0c3980', 'request_id': 'req-697c2fb8-2756-4265-9d88-e43c00d6eb5b'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:42:06.517605 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-697c2fb8-2756-4265-9d88-e43c00d6eb5b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '90b73ff5e6e44a409ce3a0220a0c3980', 'request_id': 'req-697c2fb8-2756-4265-9d88-e43c00d6eb5b'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:42:06.527509 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-697c2fb8-2756-4265-9d88-e43c00d6eb5b admin admin] The token cache is being invalidate because project 90b73ff5e6e44a409ce3a0220a0c3980 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81501) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:42:06.527790 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-697c2fb8-2756-4265-9d88-e43c00d6eb5b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:42:06.528396 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-697c2fb8-2756-4265-9d88-e43c00d6eb5b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:42:06.530595 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1282/2582] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Nov 2 07:42:06 2022] DELETE /identity/v3/projects/90b73ff5e6e44a409ce3a0220a0c3980 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:42:06.534566 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db1743a-6b19-4794-bf54-7942b2544bbf admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:42:06.535993 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-3db1743a-6b19-4794-bf54-7942b2544bbf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:42:06.549330 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db1743a-6b19-4794-bf54-7942b2544bbf admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:42:06.605601 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d0fe61-2624-450e-893b-3c9bc5ae717e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:42:06.606222 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d0fe61-2624-450e-893b-3c9bc5ae717e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:42:06.606538 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d0fe61-2624-450e-893b-3c9bc5ae717e admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:42:06.607451 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d0fe61-2624-450e-893b-3c9bc5ae717e admin admin] PATH_INFO: `/v3/users/2981733989e64de1839966e78bbcb014` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:42:06.608998 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d0fe61-2624-450e-893b-3c9bc5ae717e admin admin] RBAC: Authorizing `identity:delete_user(user_id=2981733989e64de1839966e78bbcb014)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:42:06.610927 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d0fe61-2624-450e-893b-3c9bc5ae717e admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:42:06.633125 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:42:06.662728 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-60d0fe61-2624-450e-893b-3c9bc5ae717e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2981733989e64de1839966e78bbcb014'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:42:06.671082 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-60d0fe61-2624-450e-893b-3c9bc5ae717e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2981733989e64de1839966e78bbcb014'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:42:06.671691 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-60d0fe61-2624-450e-893b-3c9bc5ae717e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2981733989e64de1839966e78bbcb014'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:42:06.678616 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-60d0fe61-2624-450e-893b-3c9bc5ae717e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2981733989e64de1839966e78bbcb014'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:42:06.679245 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-60d0fe61-2624-450e-893b-3c9bc5ae717e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2981733989e64de1839966e78bbcb014'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:42:06.697421 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1301/2583] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Nov 2 07:42:06 2022] DELETE /identity/v3/users/2981733989e64de1839966e78bbcb014 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:42:06.870412 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697c2fb8-2756-4265-9d88-e43c00d6eb5b admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:42:06.871929 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-697c2fb8-2756-4265-9d88-e43c00d6eb5b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:42:06.890043 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697c2fb8-2756-4265-9d88-e43c00d6eb5b admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:42:06.907671 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:42:06.948671 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae4f859-7059-43b6-b8ec-c28b07e36e2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:42:06.949287 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae4f859-7059-43b6-b8ec-c28b07e36e2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:42:06.949564 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae4f859-7059-43b6-b8ec-c28b07e36e2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:42:06.949833 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae4f859-7059-43b6-b8ec-c28b07e36e2d admin admin] PATH_INFO: `/v3/projects/1c7a918ce15c4294b1a5c9cf52773791` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:42:06.951105 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae4f859-7059-43b6-b8ec-c28b07e36e2d admin admin] RBAC: Authorizing `identity:delete_project(project_id=1c7a918ce15c4294b1a5c9cf52773791)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:42:06.953011 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae4f859-7059-43b6-b8ec-c28b07e36e2d admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:42:06.999373 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-3ae4f859-7059-43b6-b8ec-c28b07e36e2d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1c7a918ce15c4294b1a5c9cf52773791', 'request_id': 'req-3ae4f859-7059-43b6-b8ec-c28b07e36e2d'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:42:07.005623 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-3ae4f859-7059-43b6-b8ec-c28b07e36e2d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1c7a918ce15c4294b1a5c9cf52773791', 'request_id': 'req-3ae4f859-7059-43b6-b8ec-c28b07e36e2d'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:42:07.015171 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-3ae4f859-7059-43b6-b8ec-c28b07e36e2d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1c7a918ce15c4294b1a5c9cf52773791', 'request_id': 'req-3ae4f859-7059-43b6-b8ec-c28b07e36e2d'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:42:07.025572 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-3ae4f859-7059-43b6-b8ec-c28b07e36e2d admin admin] The token cache is being invalidate because project 1c7a918ce15c4294b1a5c9cf52773791 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81501) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:42:07.025860 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-3ae4f859-7059-43b6-b8ec-c28b07e36e2d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:42:07.026515 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-3ae4f859-7059-43b6-b8ec-c28b07e36e2d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:42:07.028005 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1283/2584] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Nov 2 07:42:06 2022] DELETE /identity/v3/projects/1c7a918ce15c4294b1a5c9cf52773791 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:42:07.033477 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa4a1a5-62d2-46d7-b3e8-069c358f18ae None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:42:07.033782 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa4a1a5-62d2-46d7-b3e8-069c358f18ae None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:42:07.034061 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa4a1a5-62d2-46d7-b3e8-069c358f18ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:42:07.054398 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-1aa4a1a5-62d2-46d7-b3e8-069c358f18ae None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:42:07.055589 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-1aa4a1a5-62d2-46d7-b3e8-069c358f18ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:42:07.115631 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1302/2585] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:42:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:42:07.123913 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc61382-0f09-4cb8-9332-e24de1f4e5c9 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:42:07.124194 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc61382-0f09-4cb8-9332-e24de1f4e5c9 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:42:07.124465 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc61382-0f09-4cb8-9332-e24de1f4e5c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:42:07.144641 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-1cc61382-0f09-4cb8-9332-e24de1f4e5c9 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:42:07.145766 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-1cc61382-0f09-4cb8-9332-e24de1f4e5c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:42:07.153635 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1284/2586] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:42:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:42:07.157866 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa4a1a5-62d2-46d7-b3e8-069c358f18ae None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:42:07.167183 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa4a1a5-62d2-46d7-b3e8-069c358f18ae None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:42:07.172011 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f78380a-716a-4829-b70c-574c8ed99087 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:42:07.172592 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f78380a-716a-4829-b70c-574c8ed99087 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:42:07.172864 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f78380a-716a-4829-b70c-574c8ed99087 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:42:07.173129 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f78380a-716a-4829-b70c-574c8ed99087 admin admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:42:07.173761 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f78380a-716a-4829-b70c-574c8ed99087 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:42:07.174056 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f78380a-716a-4829-b70c-574c8ed99087 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:42:07.175672 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f78380a-716a-4829-b70c-574c8ed99087 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:42:07.187390 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1303/2587] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Nov 2 07:42:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:42:07.191726 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc61382-0f09-4cb8-9332-e24de1f4e5c9 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:42:07.201515 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc61382-0f09-4cb8-9332-e24de1f4e5c9 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:42:07.206082 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26dc323d-3a39-4f59-8a64-466ddd9f136e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:42:07.206732 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dc323d-3a39-4f59-8a64-466ddd9f136e admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:42:07.207015 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dc323d-3a39-4f59-8a64-466ddd9f136e admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:42:07.207287 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dc323d-3a39-4f59-8a64-466ddd9f136e admin admin] PATH_INFO: `/v3/projects` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:42:07.207917 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26dc323d-3a39-4f59-8a64-466ddd9f136e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:42:07.209789 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26dc323d-3a39-4f59-8a64-466ddd9f136e admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:42:07.236828 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1285/2588] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Nov 2 07:42:07 2022] POST /identity/v3/projects => generated 373 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 07:42:07.240943 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f78380a-716a-4829-b70c-574c8ed99087 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:42:07.251017 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f78380a-716a-4829-b70c-574c8ed99087 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:42:07.307310 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd041ea-eaae-42d7-b720-d8565df65e9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:42:07.307912 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd041ea-eaae-42d7-b720-d8565df65e9c admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:42:07.308188 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd041ea-eaae-42d7-b720-d8565df65e9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:42:07.308456 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd041ea-eaae-42d7-b720-d8565df65e9c admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:42:07.309078 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd041ea-eaae-42d7-b720-d8565df65e9c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:42:07.311085 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd041ea-eaae-42d7-b720-d8565df65e9c admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:42:07.331903 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1304/2589] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Nov 2 07:42:07 2022] POST /identity/v3/users => generated 287 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:42:07.336133 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26dc323d-3a39-4f59-8a64-466ddd9f136e admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:42:07.346883 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26dc323d-3a39-4f59-8a64-466ddd9f136e admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:42:07.351441 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a93086a-aa64-4c4d-9a45-3bff68d644d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:42:07.352029 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a93086a-aa64-4c4d-9a45-3bff68d644d6 admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:42:07.352311 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a93086a-aa64-4c4d-9a45-3bff68d644d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:42:07.352584 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a93086a-aa64-4c4d-9a45-3bff68d644d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:42:07.353214 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a93086a-aa64-4c4d-9a45-3bff68d644d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:42:07.354859 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a93086a-aa64-4c4d-9a45-3bff68d644d6 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:42:07.364602 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1286/2590] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:42:07 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:42:07.368440 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd041ea-eaae-42d7-b720-d8565df65e9c admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:42:07.378544 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd041ea-eaae-42d7-b720-d8565df65e9c admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:42:07.383106 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441e54f8-cf3a-4d4f-bb4f-f5981c5876bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:42:07.383696 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441e54f8-cf3a-4d4f-bb4f-f5981c5876bb admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:42:07.383976 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441e54f8-cf3a-4d4f-bb4f-f5981c5876bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:42:07.384244 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441e54f8-cf3a-4d4f-bb4f-f5981c5876bb admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:42:07.384872 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441e54f8-cf3a-4d4f-bb4f-f5981c5876bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:42:07.386458 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441e54f8-cf3a-4d4f-bb4f-f5981c5876bb admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:42:07.397762 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1305/2591] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:42:07 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:42:07.401785 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a93086a-aa64-4c4d-9a45-3bff68d644d6 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:42:07.411947 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a93086a-aa64-4c4d-9a45-3bff68d644d6 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:42:07.416659 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7626b8cd-ddc9-423c-83ac-343de311e471 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:42:07.417296 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7626b8cd-ddc9-423c-83ac-343de311e471 admin admin] REQUEST_METHOD: `PUT` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:42:07.417579 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7626b8cd-ddc9-423c-83ac-343de311e471 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:42:07.417849 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7626b8cd-ddc9-423c-83ac-343de311e471 admin admin] PATH_INFO: `/v3/projects/7fe8d2e266ba4eaf8cdf66bf31acc537/users/c1b5bac1ae4842518f0acc9d802150a6/roles/b494fedd00f24f0b84a529e8672a156a` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:42:07.433762 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7626b8cd-ddc9-423c-83ac-343de311e471 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7fe8d2e266ba4eaf8cdf66bf31acc537, user_id=c1b5bac1ae4842518f0acc9d802150a6, role_id=b494fedd00f24f0b84a529e8672a156a)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:42:07.436983 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7626b8cd-ddc9-423c-83ac-343de311e471 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:42:07.449152 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1287/2592] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Nov 2 07:42:07 2022] PUT /identity/v3/projects/7fe8d2e266ba4eaf8cdf66bf31acc537/users/c1b5bac1ae4842518f0acc9d802150a6/roles/b494fedd00f24f0b84a529e8672a156a => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 07:42:07.460836 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06171773-739c-4ee3-a1ce-df066ab002cd None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:42:07.461151 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06171773-739c-4ee3-a1ce-df066ab002cd None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:42:07.461426 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06171773-739c-4ee3-a1ce-df066ab002cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:42:07.469510 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:42:07.489501 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-06171773-739c-4ee3-a1ce-df066ab002cd None None] MFA Rules not processed for user `c1b5bac1ae4842518f0acc9d802150a6`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:42:07.490731 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-06171773-739c-4ee3-a1ce-df066ab002cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:42:07.555321 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1306/2593] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:42:07 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:42:07.559967 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7626b8cd-ddc9-423c-83ac-343de311e471 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:42:07.607719 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:42:07.624523 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7626b8cd-ddc9-423c-83ac-343de311e471 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:42:07.630594 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968b1ab9-2bd7-4dd7-9d02-cd185d6535d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:42:07.631205 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968b1ab9-2bd7-4dd7-9d02-cd185d6535d7 admin admin] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:42:07.631482 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968b1ab9-2bd7-4dd7-9d02-cd185d6535d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:42:07.631750 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968b1ab9-2bd7-4dd7-9d02-cd185d6535d7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:42:07.632379 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968b1ab9-2bd7-4dd7-9d02-cd185d6535d7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:42:07.634302 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968b1ab9-2bd7-4dd7-9d02-cd185d6535d7 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:42:07.668657 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1288/2594] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Nov 2 07:42:07 2022] POST /identity/v3/projects => generated 375 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:42:07.673628 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06171773-739c-4ee3-a1ce-df066ab002cd None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:42:07.684292 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06171773-739c-4ee3-a1ce-df066ab002cd None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:42:07.739351 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a0672f-9cb4-4f4c-a4dc-5016a4fc2329 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:42:07.739949 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a0672f-9cb4-4f4c-a4dc-5016a4fc2329 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:42:07.740228 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a0672f-9cb4-4f4c-a4dc-5016a4fc2329 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:42:07.740509 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a0672f-9cb4-4f4c-a4dc-5016a4fc2329 admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:42:07.741150 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a0672f-9cb4-4f4c-a4dc-5016a4fc2329 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:42:07.743080 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a0672f-9cb4-4f4c-a4dc-5016a4fc2329 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:42:07.763722 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1307/2595] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Nov 2 07:42:07 2022] POST /identity/v3/users => generated 294 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:42:07.767645 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968b1ab9-2bd7-4dd7-9d02-cd185d6535d7 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:42:07.777871 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968b1ab9-2bd7-4dd7-9d02-cd185d6535d7 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:42:07.783192 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92f2482-a546-44e0-9df8-1dcfde429586 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:42:07.783808 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92f2482-a546-44e0-9df8-1dcfde429586 admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:42:07.784091 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92f2482-a546-44e0-9df8-1dcfde429586 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:42:07.784364 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92f2482-a546-44e0-9df8-1dcfde429586 admin admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:42:07.784997 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92f2482-a546-44e0-9df8-1dcfde429586 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:42:07.786807 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92f2482-a546-44e0-9df8-1dcfde429586 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:42:07.797438 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1289/2596] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:42:07 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:42:07.801419 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a0672f-9cb4-4f4c-a4dc-5016a4fc2329 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:42:07.812005 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a0672f-9cb4-4f4c-a4dc-5016a4fc2329 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:42:07.816537 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae02bdf-d076-49bf-9f51-8f6f3c00e091 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:42:07.817174 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae02bdf-d076-49bf-9f51-8f6f3c00e091 admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:42:07.817455 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae02bdf-d076-49bf-9f51-8f6f3c00e091 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:42:07.817721 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae02bdf-d076-49bf-9f51-8f6f3c00e091 admin admin] PATH_INFO: `/v3/projects/b051fbd615614fd189f6db42908ae41a/users/248ecfe12d104a3092af8ea9561228f4/roles/c4bf019fc6c44c71a6c984f6d0bab75f` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:42:07.832037 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae02bdf-d076-49bf-9f51-8f6f3c00e091 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b051fbd615614fd189f6db42908ae41a, user_id=248ecfe12d104a3092af8ea9561228f4, role_id=c4bf019fc6c44c71a6c984f6d0bab75f)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:42:07.834665 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae02bdf-d076-49bf-9f51-8f6f3c00e091 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:42:07.847601 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1308/2597] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Nov 2 07:42:07 2022] PUT /identity/v3/projects/b051fbd615614fd189f6db42908ae41a/users/248ecfe12d104a3092af8ea9561228f4/roles/c4bf019fc6c44c71a6c984f6d0bab75f => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:42:07.851468 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92f2482-a546-44e0-9df8-1dcfde429586 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:42:07.862336 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92f2482-a546-44e0-9df8-1dcfde429586 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:42:07.918352 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c650e5b-24d9-40e4-a906-42738f67ba95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:42:07.918913 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c650e5b-24d9-40e4-a906-42738f67ba95 admin admin] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:42:07.919826 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c650e5b-24d9-40e4-a906-42738f67ba95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:42:07.920097 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c650e5b-24d9-40e4-a906-42738f67ba95 admin admin] PATH_INFO: `/v3/roles` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:42:07.920723 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c650e5b-24d9-40e4-a906-42738f67ba95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:42:07.922345 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c650e5b-24d9-40e4-a906-42738f67ba95 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:42:07.931879 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1290/2598] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Nov 2 07:42:07 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:42:07.936248 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae02bdf-d076-49bf-9f51-8f6f3c00e091 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:42:07.946563 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae02bdf-d076-49bf-9f51-8f6f3c00e091 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:42:07.951118 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26017dc4-5a2d-46de-9387-712afa8ebfa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:42:07.951752 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26017dc4-5a2d-46de-9387-712afa8ebfa4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:42:07.952029 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26017dc4-5a2d-46de-9387-712afa8ebfa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:42:07.952298 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26017dc4-5a2d-46de-9387-712afa8ebfa4 admin admin] PATH_INFO: `/v3/projects/b051fbd615614fd189f6db42908ae41a/users/248ecfe12d104a3092af8ea9561228f4/roles/c4bf019fc6c44c71a6c984f6d0bab75f` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:42:07.954156 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26017dc4-5a2d-46de-9387-712afa8ebfa4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b051fbd615614fd189f6db42908ae41a, user_id=248ecfe12d104a3092af8ea9561228f4, role_id=c4bf019fc6c44c71a6c984f6d0bab75f)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:42:07.956895 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26017dc4-5a2d-46de-9387-712afa8ebfa4 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:42:07.967483 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1309/2599] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Nov 2 07:42:07 2022] PUT /identity/v3/projects/b051fbd615614fd189f6db42908ae41a/users/248ecfe12d104a3092af8ea9561228f4/roles/c4bf019fc6c44c71a6c984f6d0bab75f => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:42:07.977219 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276e3b2a-acc2-4a1b-9ade-4ab6ca825385 None None] REQUEST_METHOD: `POST` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:42:07.977505 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276e3b2a-acc2-4a1b-9ade-4ab6ca825385 None None] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:42:07.977776 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276e3b2a-acc2-4a1b-9ade-4ab6ca825385 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:42:08.005607 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.auth.core [None req-276e3b2a-acc2-4a1b-9ade-4ab6ca825385 None None] MFA Rules not processed for user `248ecfe12d104a3092af8ea9561228f4`. Rule list: `[]` (Enabled: `True`). {{(pid=81501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:42:08.006841 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-276e3b2a-acc2-4a1b-9ade-4ab6ca825385 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:42:08.071289 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1291/2600] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:42:07 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:42:08.082674 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26017dc4-5a2d-46de-9387-712afa8ebfa4 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:42:08.084453 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-26017dc4-5a2d-46de-9387-712afa8ebfa4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:42:08.137136 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:42:08.143816 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26017dc4-5a2d-46de-9387-712afa8ebfa4 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:42:08.147708 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b55984-a8df-40e8-84a7-0bcf2339bf65 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43bb326fb82e4d2c83fb2eaf91860dc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:42:08.148334 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b55984-a8df-40e8-84a7-0bcf2339bf65 service cinder] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:42:08.148614 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b55984-a8df-40e8-84a7-0bcf2339bf65 service cinder] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:42:08.148883 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b55984-a8df-40e8-84a7-0bcf2339bf65 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:42:08.204015 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b55984-a8df-40e8-84a7-0bcf2339bf65 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:42:08.205359 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b55984-a8df-40e8-84a7-0bcf2339bf65 service cinder] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:42:08.213931 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1310/2601] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:42:08 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:42:10.771954 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276e3b2a-acc2-4a1b-9ade-4ab6ca825385 None None] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:42:10.816294 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:42:10.835059 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276e3b2a-acc2-4a1b-9ade-4ab6ca825385 None None] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:42:10.839634 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8445b2b8-88b8-4f7a-88f8-4b01fcba0d63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:42:10.840252 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8445b2b8-88b8-4f7a-88f8-4b01fcba0d63 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:42:10.840529 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8445b2b8-88b8-4f7a-88f8-4b01fcba0d63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:42:10.840800 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8445b2b8-88b8-4f7a-88f8-4b01fcba0d63 admin admin] PATH_INFO: `/v3/users/c1b5bac1ae4842518f0acc9d802150a6` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:42:10.841933 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8445b2b8-88b8-4f7a-88f8-4b01fcba0d63 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c1b5bac1ae4842518f0acc9d802150a6)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:42:10.843856 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8445b2b8-88b8-4f7a-88f8-4b01fcba0d63 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:42:10.888694 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-8445b2b8-88b8-4f7a-88f8-4b01fcba0d63 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c1b5bac1ae4842518f0acc9d802150a6'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:42:10.895491 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-8445b2b8-88b8-4f7a-88f8-4b01fcba0d63 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c1b5bac1ae4842518f0acc9d802150a6'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:42:10.896253 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-8445b2b8-88b8-4f7a-88f8-4b01fcba0d63 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c1b5bac1ae4842518f0acc9d802150a6'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:42:10.903120 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-8445b2b8-88b8-4f7a-88f8-4b01fcba0d63 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c1b5bac1ae4842518f0acc9d802150a6'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:42:10.914682 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-8445b2b8-88b8-4f7a-88f8-4b01fcba0d63 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c1b5bac1ae4842518f0acc9d802150a6'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:42:10.920490 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1292/2602] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Nov 2 07:42:10 2022] DELETE /identity/v3/users/c1b5bac1ae4842518f0acc9d802150a6 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:42:10.924958 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51b40fe-3608-4d19-93bc-1bb4a1dc7069 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:42:10.925451 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51b40fe-3608-4d19-93bc-1bb4a1dc7069 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:42:10.925552 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51b40fe-3608-4d19-93bc-1bb4a1dc7069 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:42:10.944590 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-b51b40fe-3608-4d19-93bc-1bb4a1dc7069 None None] MFA Rules not processed for user `b87b731cb4ea4070ae4dad0c3c393570`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:42:10.945801 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-b51b40fe-3608-4d19-93bc-1bb4a1dc7069 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:42:11.003733 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1311/2603] 10.222.0.18 () {58 vars in 958 bytes} [Wed Nov 2 07:42:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:42:11.014099 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8445b2b8-88b8-4f7a-88f8-4b01fcba0d63 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:42:11.015614 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-8445b2b8-88b8-4f7a-88f8-4b01fcba0d63 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:42:11.072158 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8445b2b8-88b8-4f7a-88f8-4b01fcba0d63 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:42:11.075848 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6c4b42-089e-4137-b11a-c1dd9c3a3268 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3ec6ecb85f94ef59762877d9ea33ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:42:11.076446 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6c4b42-089e-4137-b11a-c1dd9c3a3268 service neutron] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:42:11.076726 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6c4b42-089e-4137-b11a-c1dd9c3a3268 service neutron] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:42:11.076995 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6c4b42-089e-4137-b11a-c1dd9c3a3268 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:42:11.085802 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6c4b42-089e-4137-b11a-c1dd9c3a3268 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:42:11.087252 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6c4b42-089e-4137-b11a-c1dd9c3a3268 service neutron] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:42:11.096514 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1293/2604] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:42:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 07:42:11.362529 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51b40fe-3608-4d19-93bc-1bb4a1dc7069 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:42:11.364039 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-b51b40fe-3608-4d19-93bc-1bb4a1dc7069 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:42:11.378287 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51b40fe-3608-4d19-93bc-1bb4a1dc7069 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:42:11.383363 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47447d9b-169a-4f35-8ce6-cd7a2a193ec4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:42:11.384181 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47447d9b-169a-4f35-8ce6-cd7a2a193ec4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:42:11.384456 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47447d9b-169a-4f35-8ce6-cd7a2a193ec4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:42:11.384727 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47447d9b-169a-4f35-8ce6-cd7a2a193ec4 admin admin] PATH_INFO: `/v3/projects/7fe8d2e266ba4eaf8cdf66bf31acc537` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:42:11.385828 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47447d9b-169a-4f35-8ce6-cd7a2a193ec4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7fe8d2e266ba4eaf8cdf66bf31acc537)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:42:11.390048 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47447d9b-169a-4f35-8ce6-cd7a2a193ec4 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:42:11.438206 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-47447d9b-169a-4f35-8ce6-cd7a2a193ec4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7fe8d2e266ba4eaf8cdf66bf31acc537', 'request_id': 'req-47447d9b-169a-4f35-8ce6-cd7a2a193ec4'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:42:11.444268 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-47447d9b-169a-4f35-8ce6-cd7a2a193ec4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7fe8d2e266ba4eaf8cdf66bf31acc537', 'request_id': 'req-47447d9b-169a-4f35-8ce6-cd7a2a193ec4'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:42:11.449574 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-47447d9b-169a-4f35-8ce6-cd7a2a193ec4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7fe8d2e266ba4eaf8cdf66bf31acc537', 'request_id': 'req-47447d9b-169a-4f35-8ce6-cd7a2a193ec4'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:42:11.461519 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-47447d9b-169a-4f35-8ce6-cd7a2a193ec4 admin admin] The token cache is being invalidate because project 7fe8d2e266ba4eaf8cdf66bf31acc537 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:42:11.461801 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-47447d9b-169a-4f35-8ce6-cd7a2a193ec4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:42:11.462443 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-47447d9b-169a-4f35-8ce6-cd7a2a193ec4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:42:11.463908 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1312/2605] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Nov 2 07:42:11 2022] DELETE /identity/v3/projects/7fe8d2e266ba4eaf8cdf66bf31acc537 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:42:11.468570 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6c4b42-089e-4137-b11a-c1dd9c3a3268 service neutron] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:42:11.477968 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6c4b42-089e-4137-b11a-c1dd9c3a3268 service neutron] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:42:11.533697 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c240342-49b4-464c-9323-6190c15ded56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:42:11.534314 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c240342-49b4-464c-9323-6190c15ded56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:42:11.534622 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c240342-49b4-464c-9323-6190c15ded56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:42:11.534894 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c240342-49b4-464c-9323-6190c15ded56 admin admin] PATH_INFO: `/v3/users/248ecfe12d104a3092af8ea9561228f4` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:42:11.535997 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c240342-49b4-464c-9323-6190c15ded56 admin admin] RBAC: Authorizing `identity:delete_user(user_id=248ecfe12d104a3092af8ea9561228f4)` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:42:11.537947 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c240342-49b4-464c-9323-6190c15ded56 admin admin] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:42:11.542245 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:42:11.585436 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-6c240342-49b4-464c-9323-6190c15ded56 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '248ecfe12d104a3092af8ea9561228f4'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:42:11.592554 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-6c240342-49b4-464c-9323-6190c15ded56 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '248ecfe12d104a3092af8ea9561228f4'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:42:11.593164 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-6c240342-49b4-464c-9323-6190c15ded56 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '248ecfe12d104a3092af8ea9561228f4'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:42:11.599928 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-6c240342-49b4-464c-9323-6190c15ded56 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '248ecfe12d104a3092af8ea9561228f4'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:42:11.611514 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.notifications [None req-6c240342-49b4-464c-9323-6190c15ded56 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '248ecfe12d104a3092af8ea9561228f4'} {{(pid=81501) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:42:11.617219 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1294/2606] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Nov 2 07:42:11 2022] DELETE /identity/v3/users/248ecfe12d104a3092af8ea9561228f4 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:42:11.883574 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47447d9b-169a-4f35-8ce6-cd7a2a193ec4 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:42:11.885052 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-47447d9b-169a-4f35-8ce6-cd7a2a193ec4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:42:11.898177 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47447d9b-169a-4f35-8ce6-cd7a2a193ec4 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:42:11.917775 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:42:11.956685 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de878e52-76d7-4bfa-af84-847ae2625d58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b87b731cb4ea4070ae4dad0c3c393570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad924d40efc4e06bdd880c155ca36a4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:42:11.957335 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de878e52-76d7-4bfa-af84-847ae2625d58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:42:11.957586 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de878e52-76d7-4bfa-af84-847ae2625d58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:42:11.957860 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de878e52-76d7-4bfa-af84-847ae2625d58 admin admin] PATH_INFO: `/v3/projects/b051fbd615614fd189f6db42908ae41a` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:42:11.959084 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de878e52-76d7-4bfa-af84-847ae2625d58 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b051fbd615614fd189f6db42908ae41a)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:42:11.961026 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de878e52-76d7-4bfa-af84-847ae2625d58 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:42:12.007297 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-de878e52-76d7-4bfa-af84-847ae2625d58 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b051fbd615614fd189f6db42908ae41a', 'request_id': 'req-de878e52-76d7-4bfa-af84-847ae2625d58'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:42:12.013314 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-de878e52-76d7-4bfa-af84-847ae2625d58 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b051fbd615614fd189f6db42908ae41a', 'request_id': 'req-de878e52-76d7-4bfa-af84-847ae2625d58'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:42:12.018707 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-de878e52-76d7-4bfa-af84-847ae2625d58 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b051fbd615614fd189f6db42908ae41a', 'request_id': 'req-de878e52-76d7-4bfa-af84-847ae2625d58'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:42:12.030616 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-de878e52-76d7-4bfa-af84-847ae2625d58 admin admin] The token cache is being invalidate because project b051fbd615614fd189f6db42908ae41a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:42:12.030904 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-de878e52-76d7-4bfa-af84-847ae2625d58 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:42:12.031521 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-de878e52-76d7-4bfa-af84-847ae2625d58 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:42:12.032996 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1313/2607] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Nov 2 07:42:11 2022] DELETE /identity/v3/projects/b051fbd615614fd189f6db42908ae41a => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 07:42:21.613907 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=24,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:42:22.027588 dv-cinder-804735-33-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=40,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:42:37.925633 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c240342-49b4-464c-9323-6190c15ded56 admin admin] Authenticating user token {{(pid=81501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:42:37.927062 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-6c240342-49b4-464c-9323-6190c15ded56 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:42:37.995423 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c240342-49b4-464c-9323-6190c15ded56 admin admin] Validating token access rules against request {{(pid=81501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:42:38.054226 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8179c1-2765-4dea-b0a8-64c5b9fc331a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2fc6a156c54d0c9d325901d0f7f6fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af04daa468b64160a6176dcab29502a7', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:42:38.054857 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8179c1-2765-4dea-b0a8-64c5b9fc331a service placement] REQUEST_METHOD: `GET` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:42:38.055164 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8179c1-2765-4dea-b0a8-64c5b9fc331a service placement] SCRIPT_NAME: `/identity` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:42:38.055476 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8179c1-2765-4dea-b0a8-64c5b9fc331a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:42:38.057153 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.fernet_utils [None req-6a8179c1-2765-4dea-b0a8-64c5b9fc331a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:42:38.068076 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8179c1-2765-4dea-b0a8-64c5b9fc331a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:42:38.069627 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8179c1-2765-4dea-b0a8-64c5b9fc331a service placement] RBAC: Authorization granted {{(pid=81501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:42:38.079830 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: [pid: 81501|app: 0|req: 1295/2608] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Nov 2 07:42:37 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:42:48.075627 dv-cinder-804735-33-iscsi devstack@keystone.service[81501]: DEBUG dbcounter [-] [81501] Writing DB stats keystone:SELECT=58 {{(pid=81501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}